[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:56:13 dist-test-slave-76rk krb5kdc[11974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:56:13 dist-test-slave-76rk krb5kdc[11974](info): set up 2 sockets
Jun 19 14:56:13 dist-test-slave-76rk krb5kdc[11974](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 14:56:14.139247 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.012s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:56:14 dist-test-slave-76rk krb5kdc[11974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:56:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:56:14.334393 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:37809
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:37809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:14.667644 11987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:14.667941 11987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:14.668033 11987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:14.675343 11987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:14.675468 11987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:14.675514 11987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:14.675573 11987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:14.675622 11987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:56:14.685719 11987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:37809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:37809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:14.687100 11987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:14.688830 11987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:14.701753 11992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:14.701673 11993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:14.701701 11995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:14.702879 11987 server_base.cc:1034] running on GCE node
I20260619 14:56:14.703786 11987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:14.705291 11987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:14.706511 11987 hybrid_clock.cc:648] HybridClock initialized: now 1781880974706467 us; error 35 us; skew 500 ppm
I20260619 14:56:14.706933 11987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:14 dist-test-slave-76rk krb5kdc[11974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880974, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:14.710374 11987 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:56:14.712146 11987 webserver.cc:460] Webserver started at http://127.11.175.254:34951/ using document root <none> and password file <none>
I20260619 14:56:14.712867 11987 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:14.713014 11987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:14.713330 11987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:14.716619 11987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "fa913ae751db4fca8a305f56e3edf90b"
format_stamp: "Formatted at 2026-06-19 14:56:14 on dist-test-slave-76rk"
server_key: "bda482e79fa28e55b07c28b8c221f71f"
server_key_iv: "a8396dbc97223e1f014e611e7d818cac"
server_key_version: "encryptionkey@0"
I20260619 14:56:14.717427 11987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "fa913ae751db4fca8a305f56e3edf90b"
format_stamp: "Formatted at 2026-06-19 14:56:14 on dist-test-slave-76rk"
server_key: "bda482e79fa28e55b07c28b8c221f71f"
server_key_iv: "a8396dbc97223e1f014e611e7d818cac"
server_key_version: "encryptionkey@0"
I20260619 14:56:14.723335 11987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 14:56:14.727201 12002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:14.728649 11987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:14.728869 11987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "fa913ae751db4fca8a305f56e3edf90b"
format_stamp: "Formatted at 2026-06-19 14:56:14 on dist-test-slave-76rk"
server_key: "bda482e79fa28e55b07c28b8c221f71f"
server_key_iv: "a8396dbc97223e1f014e611e7d818cac"
server_key_version: "encryptionkey@0"
I20260619 14:56:14.729096 11987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:14.750524 11987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:14.755864 11987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:14.756170 11987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:14.776091 11987 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:37809
I20260619 14:56:14.776132 12053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:37809 every 8 connection(s)
I20260619 14:56:14.777609 11987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:14.781616 12054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:14.783141 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 11987
I20260619 14:56:14.783465 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:14.785389 11967 external_mini_cluster.cc:1428] Setting key 978ea8cdb588a47f9a560292e80bdd35
I20260619 14:56:14.793042 12054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b: Bootstrap starting.
I20260619 14:56:14.796169 12054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:14.797214 12054 log.cc:824] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:14.800549 12054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b: No bootstrap required, opened a new log
I20260619 14:56:14.809928 12054 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa913ae751db4fca8a305f56e3edf90b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37809 } }
I20260619 14:56:14.810384 12054 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:14.810505 12054 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa913ae751db4fca8a305f56e3edf90b, State: Initialized, Role: FOLLOWER
I20260619 14:56:14.811286 12054 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa913ae751db4fca8a305f56e3edf90b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37809 } }
I20260619 14:56:14.811597 12054 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:14.811757 12054 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:14.811939 12054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:14.814404 12054 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa913ae751db4fca8a305f56e3edf90b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37809 } }
I20260619 14:56:14.814971 12054 leader_election.cc:302] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa913ae751db4fca8a305f56e3edf90b; no voters:
I20260619 14:56:14.815625 12054 leader_election.cc:288] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:14.815810 12059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:14.817078 12059 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [term 1 LEADER]: Becoming Leader. State: Replica: fa913ae751db4fca8a305f56e3edf90b, State: Running, Role: LEADER
Jun 19 14:56:14 dist-test-slave-76rk krb5kdc[11974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:14.817636 12059 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa913ae751db4fca8a305f56e3edf90b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37809 } }
I20260619 14:56:14.818084 12054 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:14.820953 12061 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa913ae751db4fca8a305f56e3edf90b. Latest consensus state: current_term: 1 leader_uuid: "fa913ae751db4fca8a305f56e3edf90b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa913ae751db4fca8a305f56e3edf90b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37809 } } }
I20260619 14:56:14.821219 12060 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa913ae751db4fca8a305f56e3edf90b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa913ae751db4fca8a305f56e3edf90b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37809 } } }
I20260619 14:56:14.821559 12060 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:14.821537 12061 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:14.833072 12066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:14.836165 12056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:14.789453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40958 (local address 127.11.175.254:37809)
0619 14:56:14.790526 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:14.790546 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:14.792233 (+ 1687us) server_negotiation.cc:407] Connection header received
0619 14:56:14.801970 (+ 9737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:14.802000 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:14.802562 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:14.803097 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:14.806064 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:14.808253 (+ 2189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:14.810358 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:14.811465 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:14.811695 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:14.819281 (+ 7586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:14.819300 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:14.819325 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:14.819389 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:14.825116 (+ 5727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:14.826910 (+ 1794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:14.826924 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:14.826937 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:14.827282 (+ 345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:14.828262 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:14.828271 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:14.828277 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:14.829356 (+ 1079us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:14.833175 (+ 3819us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:14.833650 (+ 475us) server_negotiation.cc:299] Negotiation successful
0619 14:56:14.834344 (+ 694us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":492,"threads_started":1}
I20260619 14:56:14.839007 12066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:14.850142 12066 catalog_manager.cc:1347] Generated new cluster ID: 81e0920011ef464fab89de3f253c29c3
I20260619 14:56:14.850281 12066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:14.868048 12066 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:14.869791 12066 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:14.884490 12066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa913ae751db4fca8a305f56e3edf90b: Generated new TSK 0
I20260619 14:56:14.885725 12066 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:56:14.936530 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:37809
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:15.285692 12080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:15.285981 12080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:15.286083 12080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:15.293545 12080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:15.293665 12080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:15.293792 12080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:56:15.304100 12080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:37809
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:15.305331 12080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:15.306604 12080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:15.320281 12088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:15.320271 12085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:15.320271 12086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:15.321381 12080 server_base.cc:1034] running on GCE node
I20260619 14:56:15.322006 12080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:15.323045 12080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:15.324308 12080 hybrid_clock.cc:648] HybridClock initialized: now 1781880975324203 us; error 71 us; skew 500 ppm
I20260619 14:56:15.324633 12080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:15 dist-test-slave-76rk krb5kdc[11974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880975, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:15.335109 12080 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:56:15.336941 12080 webserver.cc:460] Webserver started at http://127.11.175.193:32947/ using document root <none> and password file <none>
I20260619 14:56:15.337707 12080 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:15.337817 12080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:15.338094 12080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:15.341214 12080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "ea86b4b9c11c4416b136f9f8bc2cce4f"
format_stamp: "Formatted at 2026-06-19 14:56:15 on dist-test-slave-76rk"
server_key: "0e6a34f1f5edad87f9966d35c0733c92"
server_key_iv: "d8ab7be3182f2a2271f8bf85ceb353cc"
server_key_version: "encryptionkey@0"
I20260619 14:56:15.341966 12080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "ea86b4b9c11c4416b136f9f8bc2cce4f"
format_stamp: "Formatted at 2026-06-19 14:56:15 on dist-test-slave-76rk"
server_key: "0e6a34f1f5edad87f9966d35c0733c92"
server_key_iv: "d8ab7be3182f2a2271f8bf85ceb353cc"
server_key_version: "encryptionkey@0"
I20260619 14:56:15.347771 12080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 14:56:15.351820 12095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:15.353464 12080 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 14:56:15.353794 12080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "ea86b4b9c11c4416b136f9f8bc2cce4f"
format_stamp: "Formatted at 2026-06-19 14:56:15 on dist-test-slave-76rk"
server_key: "0e6a34f1f5edad87f9966d35c0733c92"
server_key_iv: "d8ab7be3182f2a2271f8bf85ceb353cc"
server_key_version: "encryptionkey@0"
I20260619 14:56:15.354039 12080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:15.376345 12080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:15.381485 12080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:15.381959 12080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:15.383194 12080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:15.385221 12080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:15.385309 12080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:15.385491 12080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:15.385553 12080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:15.421782 12080 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:35687
I20260619 14:56:15.421869 12207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:35687 every 8 connection(s)
I20260619 14:56:15.423218 12080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:56:15.425705 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 12080
I20260619 14:56:15.425976 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:56:15.426973 11967 external_mini_cluster.cc:1428] Setting key 24401edbdfc787add3bc471fea5916b8
Jun 19 14:56:15 dist-test-slave-76rk krb5kdc[11974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880975, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 14:56:15.447433 12210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:15.426969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:38727 (local address 127.11.175.254:37809)
0619 14:56:15.427612 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:15.427622 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:15.428421 (+ 799us) server_negotiation.cc:407] Connection header received
0619 14:56:15.430544 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:15.430550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:15.430689 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:15.430822 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:15.432577 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:15.433313 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:15.435538 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:15.436064 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:15.436176 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:15.440948 (+ 4772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:15.440954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:15.440957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:15.440995 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:15.443873 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:15.444753 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:15.444758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:15.444760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:15.444882 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:15.445477 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:15.445481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:15.445483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:15.445819 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:15.445981 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:15.446916 (+ 935us) server_negotiation.cc:299] Negotiation successful
0619 14:56:15.447111 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":284,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
I20260619 14:56:15.448590 12211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:15.426926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:37809 (local address 127.11.175.193:38727)
0619 14:56:15.428254 (+ 1328us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:15.428296 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:15.430237 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:15.431115 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:15.431125 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:15.431701 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:15.432321 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:15.432340 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:15.433497 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:15.433501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:15.435214 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:15.435233 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:15.436315 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:15.436324 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:15.436675 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:15.437489 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:15.437530 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:15.440661 (+ 3131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:15.444042 (+ 3381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:15.444052 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:15.444073 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:15.444601 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:15.445036 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:15.445041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:15.445045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:15.445311 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:15.446036 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:15.446047 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:15.446428 (+ 381us) client_negotiation.cc:769] Sending connection context
0619 14:56:15.446783 (+ 355us) client_negotiation.cc:240] Negotiation successful
0619 14:56:15.447296 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"thread_start_us":799,"threads_started":1}
I20260619 14:56:15.450412 12208 heartbeater.cc:347] Connected to a master server at 127.11.175.254:37809
I20260619 14:56:15.450829 12208 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:15.451666 12208 heartbeater.cc:511] Master 127.11.175.254:37809 requested a full tablet report, sending...
I20260619 14:56:15.454274 12019 ts_manager.cc:194] Registered new tserver with Master: ea86b4b9c11c4416b136f9f8bc2cce4f (127.11.175.193:35687)
I20260619 14:56:15.456605 12019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:38727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:56:15.473779 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:37809
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:15.824703 12214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:15.824998 12214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:15.825096 12214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:15.832463 12214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:15.832576 12214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:15.832698 12214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:56:15.843746 12214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:37809
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:15.844976 12214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:15.846356 12214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:15.860687 12219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:15.860634 12222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:15.860556 12220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:15.861683 12214 server_base.cc:1034] running on GCE node
I20260619 14:56:15.862484 12214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:15.863571 12214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:15.864827 12214 hybrid_clock.cc:648] HybridClock initialized: now 1781880975864774 us; error 39 us; skew 500 ppm
I20260619 14:56:15.865190 12214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:15 dist-test-slave-76rk krb5kdc[11974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880975, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:15.868645 12214 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:56:15.870249 12214 webserver.cc:460] Webserver started at http://127.11.175.194:38777/ using document root <none> and password file <none>
I20260619 14:56:15.871029 12214 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:15.871140 12214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:15.871534 12214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:15.874545 12214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "a9b8b35abcd84b308e25e174b63ef0f9"
format_stamp: "Formatted at 2026-06-19 14:56:15 on dist-test-slave-76rk"
server_key: "348ab2e8a65f8b63b1d40140e8f5ba09"
server_key_iv: "1a558c650c6dce002bacb026ccc73fce"
server_key_version: "encryptionkey@0"
I20260619 14:56:15.875389 12214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "a9b8b35abcd84b308e25e174b63ef0f9"
format_stamp: "Formatted at 2026-06-19 14:56:15 on dist-test-slave-76rk"
server_key: "348ab2e8a65f8b63b1d40140e8f5ba09"
server_key_iv: "1a558c650c6dce002bacb026ccc73fce"
server_key_version: "encryptionkey@0"
I20260619 14:56:15.881279 12214 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 14:56:15.885377 12229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:15.887068 12214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 14:56:15.887315 12214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "a9b8b35abcd84b308e25e174b63ef0f9"
format_stamp: "Formatted at 2026-06-19 14:56:15 on dist-test-slave-76rk"
server_key: "348ab2e8a65f8b63b1d40140e8f5ba09"
server_key_iv: "1a558c650c6dce002bacb026ccc73fce"
server_key_version: "encryptionkey@0"
I20260619 14:56:15.887571 12214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:15.920065 12214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:15.924883 12214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:15.925195 12214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:15.926271 12214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:15.927773 12214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:15.927888 12214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:15.928030 12214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:15.928109 12214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:15.967389 12214 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:39497
I20260619 14:56:15.967520 12341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:39497 every 8 connection(s)
I20260619 14:56:15.968842 12214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:56:15.973973 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 12214
I20260619 14:56:15.974176 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:56:15.974612 11967 external_mini_cluster.cc:1428] Setting key 1ea098c28c75a1499bfe2b6ac2df9023
Jun 19 14:56:15 dist-test-slave-76rk krb5kdc[11974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880975, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:15.990574 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:15.971840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:49309 (local address 127.11.175.254:37809)
0619 14:56:15.972339 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:15.972344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:15.973324 (+ 980us) server_negotiation.cc:407] Connection header received
0619 14:56:15.975497 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:15.975503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:15.975605 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:15.975730 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:15.977427 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:15.978192 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:15.979904 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:15.980350 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:15.980495 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:15.984456 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:15.984462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:15.984465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:15.984504 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:15.987249 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:15.988082 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:15.988086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:15.988089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:15.988235 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:15.988706 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:15.988716 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:15.988718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:15.988993 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:15.989156 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:15.989932 (+ 776us) server_negotiation.cc:299] Negotiation successful
0619 14:56:15.990173 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":266,"threads_started":1}
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
I20260619 14:56:15.991787 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:15.972348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:37809 (local address 127.11.175.194:49309)
0619 14:56:15.973130 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:15.973178 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:15.975174 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:15.975996 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:15.976006 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:15.976565 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:15.977234 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:15.977255 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:15.978374 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:15.978378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:15.979750 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:15.979763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:15.980592 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:15.980599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:15.980859 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:15.981583 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:15.981606 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:15.984173 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:15.987429 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:15.987438 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:15.987465 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:15.987925 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:15.988356 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:15.988361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:15.988365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:15.988578 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:15.989131 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:15.989140 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:15.989520 (+ 380us) client_negotiation.cc:769] Sending connection context
0619 14:56:15.989828 (+ 308us) client_negotiation.cc:240] Negotiation successful
0619 14:56:15.990422 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":340,"threads_started":1}
I20260619 14:56:15.993813 12342 heartbeater.cc:347] Connected to a master server at 127.11.175.254:37809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
I20260619 14:56:15.994249 12342 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:15.995335 12342 heartbeater.cc:511] Master 127.11.175.254:37809 requested a full tablet report, sending...
I20260619 14:56:15.997013 12019 ts_manager.cc:194] Registered new tserver with Master: a9b8b35abcd84b308e25e174b63ef0f9 (127.11.175.194:39497)
I20260619 14:56:15.997860 12019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:49309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:56:16.018139 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:37809
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:16.370718 12348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:16.371023 12348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:16.371122 12348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:16.378593 12348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:16.378741 12348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:16.378862 12348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:56:16.389358 12348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:37809
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:16.390703 12348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:16.392220 12348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:16.406508 12356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:16.406580 12354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:16.406807 12353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:16.407653 12348 server_base.cc:1034] running on GCE node
I20260619 14:56:16.408461 12348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:16.409597 12348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:16.410848 12348 hybrid_clock.cc:648] HybridClock initialized: now 1781880976410802 us; error 43 us; skew 500 ppm
I20260619 14:56:16.411254 12348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:16 dist-test-slave-76rk krb5kdc[11974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880976, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:16.414985 12348 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:56:16.417016 12348 webserver.cc:460] Webserver started at http://127.11.175.195:39857/ using document root <none> and password file <none>
I20260619 14:56:16.417851 12348 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:16.418020 12348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:16.418438 12348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:16.421911 12348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "0787a5fbb59e4809b274c13e41517d5b"
format_stamp: "Formatted at 2026-06-19 14:56:16 on dist-test-slave-76rk"
server_key: "d623722ca4f51c7f9c26a828a9035c8b"
server_key_iv: "f79ae04025379d1090aa96aee160f857"
server_key_version: "encryptionkey@0"
I20260619 14:56:16.422904 12348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "0787a5fbb59e4809b274c13e41517d5b"
format_stamp: "Formatted at 2026-06-19 14:56:16 on dist-test-slave-76rk"
server_key: "d623722ca4f51c7f9c26a828a9035c8b"
server_key_iv: "f79ae04025379d1090aa96aee160f857"
server_key_version: "encryptionkey@0"
I20260619 14:56:16.429129 12348 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260619 14:56:16.433372 12363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:16.435212 12348 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 14:56:16.435453 12348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "0787a5fbb59e4809b274c13e41517d5b"
format_stamp: "Formatted at 2026-06-19 14:56:16 on dist-test-slave-76rk"
server_key: "d623722ca4f51c7f9c26a828a9035c8b"
server_key_iv: "f79ae04025379d1090aa96aee160f857"
server_key_version: "encryptionkey@0"
I20260619 14:56:16.435714 12348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:16.452910 12348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:16.457701 12348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:16.458007 12348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:16.459185 12348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:16.460614 12208 heartbeater.cc:503] Master 127.11.175.254:37809 was elected leader, sending a full tablet report...
I20260619 14:56:16.460664 12348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:16.460783 12348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:16.460953 12348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:16.461028 12348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:16.497502 12348 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:39911
I20260619 14:56:16.497553 12475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:39911 every 8 connection(s)
I20260619 14:56:16.498984 12348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:56:16.507596 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 12348
I20260619 14:56:16.507802 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:56:16.508412 11967 external_mini_cluster.cc:1428] Setting key fc0958068edf3655b60c8202832976a1
Jun 19 14:56:16 dist-test-slave-76rk krb5kdc[11974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880976, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:16.520802 12478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:16.501599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:54249 (local address 127.11.175.254:37809)
0619 14:56:16.502061 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:16.502068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:16.503234 (+ 1166us) server_negotiation.cc:407] Connection header received
0619 14:56:16.505073 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:16.505078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:16.505165 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:16.505275 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:16.506959 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.508154 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.510096 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.510501 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.510613 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:16.514762 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:16.514768 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:16.514772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:16.514810 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:16.517704 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.518531 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.518535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.518537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.518725 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.519137 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.519140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.519142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.519417 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:16.519562 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:16.520355 (+ 793us) server_negotiation.cc:299] Negotiation successful
0619 14:56:16.520526 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":209,"threads_started":1}
I20260619 14:56:16.521967 12479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:16.502231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:37809 (local address 127.11.175.195:54249)
0619 14:56:16.503061 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:16.503105 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:16.504779 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:16.505500 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:16.505509 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:16.506104 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:16.506779 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:16.506799 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.508328 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.508333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:16.509913 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:16.509931 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.510807 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.510814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:16.511145 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:16.511951 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:16.511983 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:16.514460 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:16.517861 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:16.517870 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:16.517884 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:16.518363 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:16.518831 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:16.518835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:16.518837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:16.519035 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:16.519528 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:16.519534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:16.520004 (+ 470us) client_negotiation.cc:769] Sending connection context
0619 14:56:16.520239 (+ 235us) client_negotiation.cc:240] Negotiation successful
0619 14:56:16.520630 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":327,"threads_started":1}
I20260619 14:56:16.523733 12476 heartbeater.cc:347] Connected to a master server at 127.11.175.254:37809
I20260619 14:56:16.524142 12476 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:16.525090 12476 heartbeater.cc:511] Master 127.11.175.254:37809 requested a full tablet report, sending...
I20260619 14:56:16.526728 12019 ts_manager.cc:194] Registered new tserver with Master: 0787a5fbb59e4809b274c13e41517d5b (127.11.175.195:39911)
I20260619 14:56:16.527596 12019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:54249
I20260619 14:56:16.537958 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 14:56:16 dist-test-slave-76rk krb5kdc[11974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880976, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 14:56:16 dist-test-slave-76rk krb5kdc[11974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880976, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:56:16.587420 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:16.570021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53868 (local address 127.11.175.193:35687)
0619 14:56:16.570788 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:16.570796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:16.570825 (+ 29us) server_negotiation.cc:407] Connection header received
0619 14:56:16.571048 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:16.571056 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:16.571318 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:16.571463 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:16.573119 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.574130 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.575586 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.576195 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.576345 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:16.580620 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:16.580633 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:16.580649 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:16.580691 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:16.583892 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.584784 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.584792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.584795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.584945 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.585460 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.585464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.585466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.585871 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:16.586263 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:16.586620 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 14:56:16.586983 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":375,"threads_started":1}
W20260619 14:56:16.588893 12123 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53868
Jun 19 14:56:16 dist-test-slave-76rk krb5kdc[11974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 14:56:16 dist-test-slave-76rk krb5kdc[11974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:56:16.640236 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:16.623843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.11.175.193:35687)
0619 14:56:16.624084 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:16.624090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:16.624238 (+ 148us) server_negotiation.cc:407] Connection header received
0619 14:56:16.624352 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:16.624357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:16.624458 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:16.624553 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:16.626083 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.627122 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.628829 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.629305 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.629426 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:16.634266 (+ 4840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:16.634271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:16.634275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:16.634317 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:16.637469 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.638206 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.638211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.638214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.638360 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.638774 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.638778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.638780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.639085 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:16.639313 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:16.639603 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 14:56:16.639852 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260619 14:56:16.642277 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 12080
I20260619 14:56:16.655997 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:16.643403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53890 (local address 127.11.175.193:35687)
0619 14:56:16.643570 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:16.643574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:16.644015 (+ 441us) server_negotiation.cc:407] Connection header received
0619 14:56:16.644115 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:16.644119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:16.644186 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:16.644268 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:16.645555 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.646327 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.647749 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.648170 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.648268 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:16.650599 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:16.650604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:16.650607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:16.650702 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:16.653446 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.654189 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.654196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.654198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.654341 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.654753 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.654756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.654758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.655019 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:16.655208 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:16.655529 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 14:56:16.655735 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 14:56:16.656888 12203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:16.742508 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 12080
I20260619 14:56:16.760036 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 12214
Jun 19 14:56:16 dist-test-slave-76rk krb5kdc[11974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:56:16.778182 12491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:16.761097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39220 (local address 127.11.175.194:39497)
0619 14:56:16.761689 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:16.761697 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:16.761723 (+ 26us) server_negotiation.cc:407] Connection header received
0619 14:56:16.761794 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:16.761801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:16.762014 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:16.762230 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:16.763614 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.764613 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.765885 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.766461 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.766627 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:16.771328 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:16.771343 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:16.771359 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:16.771399 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:16.774865 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.775647 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.775658 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.775661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.775833 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.776345 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.776353 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.776357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.776821 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:16.777137 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:16.777458 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 14:56:16.777737 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":271,"threads_started":1}
I20260619 14:56:16.779634 12337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:16.865410 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 12214
I20260619 14:56:16.884038 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 12348
Jun 19 14:56:16 dist-test-slave-76rk krb5kdc[11974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:56:16.902681 12493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:16.885222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48432 (local address 127.11.175.195:39911)
0619 14:56:16.885983 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:16.885989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:16.886020 (+ 31us) server_negotiation.cc:407] Connection header received
0619 14:56:16.886084 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:16.886093 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:16.886320 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:16.886459 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:16.887896 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.888971 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.890345 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:16.891252 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:16.891387 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:16.896107 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:16.896123 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:16.896139 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:16.896182 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:16.899307 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.900350 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.900358 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.900361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.900510 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:16.900976 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:16.900980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:16.900982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:16.901376 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:16.901626 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:16.901984 (+ 358us) server_negotiation.cc:299] Negotiation successful
0619 14:56:16.902201 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":413,"threads_started":1}
I20260619 14:56:16.904062 12471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:16.988494 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 12348
I20260619 14:56:17.004698 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 11987
I20260619 14:56:17.005975 12049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:17.114611 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 11987
2026-06-19T14:56:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4082 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:56:17 dist-test-slave-76rk krb5kdc[12498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:56:17 dist-test-slave-76rk krb5kdc[12498](info): set up 2 sockets
Jun 19 14:56:17 dist-test-slave-76rk krb5kdc[12498](info): commencing operation
krb5kdc: starting...
W20260619 14:56:18.210801 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:56:18 dist-test-slave-76rk krb5kdc[12498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:56:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:56:18.409648 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:34471
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:34471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:18.747135 12511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:18.747469 12511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:18.747637 12511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:18.755404 12511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:18.755532 12511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:18.755621 12511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:18.755707 12511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:18.755800 12511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:56:18.766445 12511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:34471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:34471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:18.767871 12511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:18.769142 12511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:18.781643 12516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:18.781716 12517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:18.781679 12519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:18.782718 12511 server_base.cc:1034] running on GCE node
I20260619 14:56:18.783666 12511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:18.785197 12511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:18.786461 12511 hybrid_clock.cc:648] HybridClock initialized: now 1781880978786381 us; error 42 us; skew 500 ppm
I20260619 14:56:18.786887 12511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:18 dist-test-slave-76rk krb5kdc[12498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880978, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:18.790199 12511 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:56:18.792084 12511 webserver.cc:460] Webserver started at http://127.11.175.254:34699/ using document root <none> and password file <none>
I20260619 14:56:18.792820 12511 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:18.792973 12511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:18.793319 12511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:18.796931 12511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "0f37f55b11bc4d40a440be03351c33fb"
format_stamp: "Formatted at 2026-06-19 14:56:18 on dist-test-slave-76rk"
server_key: "6f751f0df482fbf89a501776efc0157b"
server_key_iv: "1f9b34a51cc6f674d82f82afeb04f504"
server_key_version: "encryptionkey@0"
I20260619 14:56:18.797834 12511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "0f37f55b11bc4d40a440be03351c33fb"
format_stamp: "Formatted at 2026-06-19 14:56:18 on dist-test-slave-76rk"
server_key: "6f751f0df482fbf89a501776efc0157b"
server_key_iv: "1f9b34a51cc6f674d82f82afeb04f504"
server_key_version: "encryptionkey@0"
I20260619 14:56:18.803812 12511 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260619 14:56:18.807919 12526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:18.809346 12511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 14:56:18.809566 12511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "0f37f55b11bc4d40a440be03351c33fb"
format_stamp: "Formatted at 2026-06-19 14:56:18 on dist-test-slave-76rk"
server_key: "6f751f0df482fbf89a501776efc0157b"
server_key_iv: "1f9b34a51cc6f674d82f82afeb04f504"
server_key_version: "encryptionkey@0"
I20260619 14:56:18.809789 12511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:18.835446 12511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:18.840308 12511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:18.840615 12511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:18.860391 12511 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:34471
I20260619 14:56:18.860431 12577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:34471 every 8 connection(s)
I20260619 14:56:18.861927 12511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:18.867425 12578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:18.869500 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 12511
I20260619 14:56:18.869772 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:18.870534 11967 external_mini_cluster.cc:1428] Setting key 455f3527dea8d1d2b07a3d5cc5ea3f51
I20260619 14:56:18.883286 12578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb: Bootstrap starting.
Jun 19 14:56:18 dist-test-slave-76rk krb5kdc[12498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:18.886955 12578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:18.888167 12578 log.cc:824] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:18.891505 12578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb: No bootstrap required, opened a new log
I20260619 14:56:18.895344 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:18.873648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.11.175.254:34471)
0619 14:56:18.874575 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:18.874592 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:18.874689 (+ 97us) server_negotiation.cc:407] Connection header received
0619 14:56:18.876515 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:18.876544 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:18.876998 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:18.877453 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:18.879139 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:18.880595 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:18.881907 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:18.882610 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:18.882854 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:18.887210 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:18.887219 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:18.887235 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:18.887287 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:18.890840 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:18.891589 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:18.891596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:18.891603 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:18.891826 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:18.892219 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:18.892224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:18.892227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:18.892778 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:18.893107 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:18.893693 (+ 586us) server_negotiation.cc:299] Negotiation successful
0619 14:56:18.894161 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":414,"threads_started":1}
I20260619 14:56:18.900103 12578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f37f55b11bc4d40a440be03351c33fb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34471 } }
I20260619 14:56:18.900545 12578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:18.900671 12578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f37f55b11bc4d40a440be03351c33fb, State: Initialized, Role: FOLLOWER
I20260619 14:56:18.901434 12578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f37f55b11bc4d40a440be03351c33fb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34471 } }
I20260619 14:56:18.901793 12578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:18.901954 12578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:18.902143 12578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:18.905121 12578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f37f55b11bc4d40a440be03351c33fb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34471 } }
I20260619 14:56:18.905814 12578 leader_election.cc:302] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f37f55b11bc4d40a440be03351c33fb; no voters:
I20260619 14:56:18.906584 12578 leader_election.cc:288] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:18.906757 12583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:18.907449 12583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [term 1 LEADER]: Becoming Leader. State: Replica: 0f37f55b11bc4d40a440be03351c33fb, State: Running, Role: LEADER
I20260619 14:56:18.908077 12583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f37f55b11bc4d40a440be03351c33fb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34471 } }
I20260619 14:56:18.908867 12578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:18.911180 12584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f37f55b11bc4d40a440be03351c33fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f37f55b11bc4d40a440be03351c33fb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34471 } } }
I20260619 14:56:18.913152 12584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:18.913384 12585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f37f55b11bc4d40a440be03351c33fb. Latest consensus state: current_term: 1 leader_uuid: "0f37f55b11bc4d40a440be03351c33fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f37f55b11bc4d40a440be03351c33fb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34471 } } }
I20260619 14:56:18.914156 12585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:18.917125 12596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:18.922003 12596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:18.931250 12596 catalog_manager.cc:1347] Generated new cluster ID: d38f10df506a4554ba89e1b42cae61a7
I20260619 14:56:18.931382 12596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:18.945986 12596 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:18.947471 12596 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:18.953088 12596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f37f55b11bc4d40a440be03351c33fb: Generated new TSK 0
I20260619 14:56:18.953903 12596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:56:19.008209 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:34471
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:19.361114 12604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:19.361424 12604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:19.361598 12604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:19.370539 12604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:19.370718 12604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:19.370936 12604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:56:19.381749 12604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:34471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:19.383235 12604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:19.384606 12604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:19.398442 12609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:19.398567 12612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:19.398494 12610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:19.399922 12604 server_base.cc:1034] running on GCE node
I20260619 14:56:19.400617 12604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:19.401711 12604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:19.402926 12604 hybrid_clock.cc:648] HybridClock initialized: now 1781880979402853 us; error 62 us; skew 500 ppm
I20260619 14:56:19.403239 12604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:19 dist-test-slave-76rk krb5kdc[12498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880979, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:19.406517 12604 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:56:19.408056 12604 webserver.cc:460] Webserver started at http://127.11.175.193:42959/ using document root <none> and password file <none>
I20260619 14:56:19.408793 12604 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:19.408922 12604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:19.409257 12604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:19.412489 12604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "51b6667477a0415bbb377fe80ca7b2bd"
format_stamp: "Formatted at 2026-06-19 14:56:19 on dist-test-slave-76rk"
server_key: "af3ec21a6e7426c128b92ddc34ec6c63"
server_key_iv: "67a76896ecc8554ccb1b8d237d84b2a1"
server_key_version: "encryptionkey@0"
I20260619 14:56:19.413298 12604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "51b6667477a0415bbb377fe80ca7b2bd"
format_stamp: "Formatted at 2026-06-19 14:56:19 on dist-test-slave-76rk"
server_key: "af3ec21a6e7426c128b92ddc34ec6c63"
server_key_iv: "67a76896ecc8554ccb1b8d237d84b2a1"
server_key_version: "encryptionkey@0"
I20260619 14:56:19.418923 12604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 14:56:19.422823 12619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:19.424577 12604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:19.424738 12604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "51b6667477a0415bbb377fe80ca7b2bd"
format_stamp: "Formatted at 2026-06-19 14:56:19 on dist-test-slave-76rk"
server_key: "af3ec21a6e7426c128b92ddc34ec6c63"
server_key_iv: "67a76896ecc8554ccb1b8d237d84b2a1"
server_key_version: "encryptionkey@0"
I20260619 14:56:19.424937 12604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:19.461268 12604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:19.466017 12604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:19.466284 12604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:19.467319 12604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:19.468732 12604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:19.468811 12604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:19.468992 12604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:19.469055 12604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:19.503931 12604 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:44493
I20260619 14:56:19.504030 12731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:44493 every 8 connection(s)
I20260619 14:56:19.505275 12604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:56:19.509914 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 12604
I20260619 14:56:19.510179 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:56:19.510928 11967 external_mini_cluster.cc:1428] Setting key 8514e830445e0ceb029307f61ec64649
Jun 19 14:56:19 dist-test-slave-76rk krb5kdc[12498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880979, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 14:56:19.529385 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:19.508245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:33783 (local address 127.11.175.254:34471)
0619 14:56:19.508714 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:19.508719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:19.509854 (+ 1135us) server_negotiation.cc:407] Connection header received
0619 14:56:19.512082 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:19.512088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:19.512226 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:19.512365 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:19.514556 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:19.515894 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:19.517818 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:19.518406 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:19.518508 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:19.523217 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:19.523223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:19.523226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:19.523269 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:19.526013 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:19.526904 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:19.526911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:19.526915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:19.527046 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:19.527500 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:19.527505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:19.527508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:19.527790 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:19.527941 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:19.528761 (+ 820us) server_negotiation.cc:299] Negotiation successful
0619 14:56:19.528999 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":252,"threads_started":1}
I20260619 14:56:19.530612 12735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:19.508859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:34471 (local address 127.11.175.193:33783)
0619 14:56:19.509683 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:19.509723 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:19.511786 (+ 2063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:19.512707 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:19.512721 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:19.513628 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:19.514346 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:19.514382 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:19.516096 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:19.516109 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:19.517631 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:19.517646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:19.518704 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:19.518715 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:19.519077 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:19.519967 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:19.520008 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:19.522943 (+ 2935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:19.526188 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:19.526199 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:19.526218 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:19.526775 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:19.527166 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:19.527172 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:19.527175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:19.527381 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:19.527929 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:19.527938 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:19.528319 (+ 381us) client_negotiation.cc:769] Sending connection context
0619 14:56:19.528641 (+ 322us) client_negotiation.cc:240] Negotiation successful
0619 14:56:19.529224 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":373,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
I20260619 14:56:19.532500 12732 heartbeater.cc:347] Connected to a master server at 127.11.175.254:34471
I20260619 14:56:19.532851 12732 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:19.533612 12732 heartbeater.cc:511] Master 127.11.175.254:34471 requested a full tablet report, sending...
I20260619 14:56:19.536052 12543 ts_manager.cc:194] Registered new tserver with Master: 51b6667477a0415bbb377fe80ca7b2bd (127.11.175.193:44493)
I20260619 14:56:19.538458 12543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:33783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:56:19.557770 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:34471
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:19.891494 12738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:19.891813 12738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:19.891969 12738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:19.899602 12738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:19.899729 12738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:19.899928 12738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:56:19.910508 12738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:34471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:19.911885 12738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:19.913437 12738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:19.929091 12744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:19.929453 12746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:19.929683 12738 server_base.cc:1034] running on GCE node
W20260619 14:56:19.929754 12743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:19.930501 12738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:19.931602 12738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:19.932991 12738 hybrid_clock.cc:648] HybridClock initialized: now 1781880979932927 us; error 52 us; skew 500 ppm
I20260619 14:56:19.933336 12738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:19 dist-test-slave-76rk krb5kdc[12498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880979, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:19.937191 12738 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:56:19.939014 12738 webserver.cc:460] Webserver started at http://127.11.175.194:44895/ using document root <none> and password file <none>
I20260619 14:56:19.939772 12738 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:19.939888 12738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:19.940168 12738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:19.943452 12738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "02935914c52b4149b4b179b934ca7d2c"
format_stamp: "Formatted at 2026-06-19 14:56:19 on dist-test-slave-76rk"
server_key: "40f8ec559735f4078603f6d162a71e0c"
server_key_iv: "9267bc5e6b7c2fb5c495e4a72b216c4e"
server_key_version: "encryptionkey@0"
I20260619 14:56:19.944267 12738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "02935914c52b4149b4b179b934ca7d2c"
format_stamp: "Formatted at 2026-06-19 14:56:19 on dist-test-slave-76rk"
server_key: "40f8ec559735f4078603f6d162a71e0c"
server_key_iv: "9267bc5e6b7c2fb5c495e4a72b216c4e"
server_key_version: "encryptionkey@0"
I20260619 14:56:19.950030 12738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 14:56:19.953984 12753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:19.955583 12738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:56:19.955761 12738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "02935914c52b4149b4b179b934ca7d2c"
format_stamp: "Formatted at 2026-06-19 14:56:19 on dist-test-slave-76rk"
server_key: "40f8ec559735f4078603f6d162a71e0c"
server_key_iv: "9267bc5e6b7c2fb5c495e4a72b216c4e"
server_key_version: "encryptionkey@0"
I20260619 14:56:19.955953 12738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:19.973785 12738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:19.979418 12738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:19.979734 12738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:19.980749 12738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:19.982277 12738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:19.982350 12738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:19.982523 12738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:19.982583 12738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:20.022083 12738 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:43299
I20260619 14:56:20.022121 12865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:43299 every 8 connection(s)
I20260619 14:56:20.023625 12738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:56:20.027487 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 12738
I20260619 14:56:20.027726 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:56:20.028360 11967 external_mini_cluster.cc:1428] Setting key 6ad2c67fbd1fde2dac29dcfb488d3426
Jun 19 14:56:20 dist-test-slave-76rk krb5kdc[12498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880979, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
I20260619 14:56:20.049625 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.026715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:51255 (local address 127.11.175.254:34471)
0619 14:56:20.026981 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:20.026987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:20.028498 (+ 1511us) server_negotiation.cc:407] Connection header received
0619 14:56:20.031222 (+ 2724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:20.031228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:20.031312 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:20.031429 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:20.033183 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.034373 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.036217 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.036669 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.036923 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.041745 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:20.041757 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:20.041766 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:20.041820 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:20.045898 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:20.046945 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:20.046951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:20.046955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:20.047125 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:20.047657 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:20.047663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:20.047667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:20.048108 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:20.048295 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:20.049116 (+ 821us) server_negotiation.cc:299] Negotiation successful
0619 14:56:20.049374 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260619 14:56:20.050853 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.027301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:34471 (local address 127.11.175.194:51255)
0619 14:56:20.028288 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:20.028335 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:20.030891 (+ 2556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:20.031694 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:20.031703 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:20.032323 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:20.032972 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.032992 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.034567 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.034571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.036040 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.036055 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.037058 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.037067 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.037549 (+ 482us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.038426 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:20.038463 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:20.041395 (+ 2932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:20.046118 (+ 4723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:20.046136 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:20.046159 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:20.046780 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:20.047288 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:20.047292 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:20.047294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:20.047507 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:20.048288 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:20.048294 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:20.048684 (+ 390us) client_negotiation.cc:769] Sending connection context
0619 14:56:20.048951 (+ 267us) client_negotiation.cc:240] Negotiation successful
0619 14:56:20.049476 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":470,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
I20260619 14:56:20.053339 12866 heartbeater.cc:347] Connected to a master server at 127.11.175.254:34471
I20260619 14:56:20.053790 12866 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:20.054980 12866 heartbeater.cc:511] Master 127.11.175.254:34471 requested a full tablet report, sending...
I20260619 14:56:20.057394 12543 ts_manager.cc:194] Registered new tserver with Master: 02935914c52b4149b4b179b934ca7d2c (127.11.175.194:43299)
I20260619 14:56:20.058424 12543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:51255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:56:20.083755 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:34471
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:20.433354 12871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:20.433638 12871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:20.433811 12871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:20.441150 12871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:20.441277 12871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:20.441453 12871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:56:20.451548 12871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:34471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:20.452845 12871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:20.454203 12871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:20.468012 12876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:20.468036 12877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:20.467960 12879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:20.468633 12871 server_base.cc:1034] running on GCE node
I20260619 14:56:20.469314 12871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:20.470362 12871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:20.471608 12871 hybrid_clock.cc:648] HybridClock initialized: now 1781880980471492 us; error 109 us; skew 500 ppm
I20260619 14:56:20.472090 12871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:20 dist-test-slave-76rk krb5kdc[12498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880980, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:20.480504 12871 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:56:20.482229 12871 webserver.cc:460] Webserver started at http://127.11.175.195:41929/ using document root <none> and password file <none>
I20260619 14:56:20.483074 12871 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:20.483223 12871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:20.483543 12871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:20.486581 12871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "1274d0d6c5154c49af980b4edcb4f640"
format_stamp: "Formatted at 2026-06-19 14:56:20 on dist-test-slave-76rk"
server_key: "0b8ec674799947d3e783d08cb56fcaba"
server_key_iv: "7fc2bf2fba76a85f9b6f92b62217cbed"
server_key_version: "encryptionkey@0"
I20260619 14:56:20.487462 12871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "1274d0d6c5154c49af980b4edcb4f640"
format_stamp: "Formatted at 2026-06-19 14:56:20 on dist-test-slave-76rk"
server_key: "0b8ec674799947d3e783d08cb56fcaba"
server_key_iv: "7fc2bf2fba76a85f9b6f92b62217cbed"
server_key_version: "encryptionkey@0"
I20260619 14:56:20.492916 12871 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:56:20.496730 12886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:20.498131 12871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 14:56:20.498322 12871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "1274d0d6c5154c49af980b4edcb4f640"
format_stamp: "Formatted at 2026-06-19 14:56:20 on dist-test-slave-76rk"
server_key: "0b8ec674799947d3e783d08cb56fcaba"
server_key_iv: "7fc2bf2fba76a85f9b6f92b62217cbed"
server_key_version: "encryptionkey@0"
I20260619 14:56:20.498538 12871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:20.534099 12871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:20.538932 12871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:20.539280 12871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:20.540413 12871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:20.541923 12732 heartbeater.cc:503] Master 127.11.175.254:34471 was elected leader, sending a full tablet report...
I20260619 14:56:20.542021 12871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:20.542111 12871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:20.542258 12871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:20.542330 12871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:20.580283 12871 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:35103
I20260619 14:56:20.580384 12998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:35103 every 8 connection(s)
I20260619 14:56:20.581799 12871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:56:20.584410 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 12871
I20260619 14:56:20.584649 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:56:20.585209 11967 external_mini_cluster.cc:1428] Setting key 21a4ec5e53b36df9cda9faa69f45e090
Jun 19 14:56:20 dist-test-slave-76rk krb5kdc[12498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880980, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:20.604148 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.584818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:55665 (local address 127.11.175.254:34471)
0619 14:56:20.585367 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:20.585374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:20.586717 (+ 1343us) server_negotiation.cc:407] Connection header received
0619 14:56:20.588764 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:20.588768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:20.588838 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:20.588930 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:20.590823 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.591611 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.593317 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.593776 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.593886 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.597605 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:20.597610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:20.597614 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:20.597692 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:20.600848 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:20.601661 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:20.601668 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:20.601671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:20.601806 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:20.602287 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:20.602292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:20.602296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:20.602579 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:20.602778 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:20.603679 (+ 901us) server_negotiation.cc:299] Negotiation successful
0619 14:56:20.603867 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":288,"threads_started":1}
I20260619 14:56:20.605197 13002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.585585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:34471 (local address 127.11.175.195:55665)
0619 14:56:20.586503 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:20.586559 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:20.588472 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:20.589232 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:20.589241 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:20.589899 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:20.590543 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.590563 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.591806 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.591810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.593159 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.593172 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.593997 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.594002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.594281 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.595052 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:20.595083 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:20.597350 (+ 2267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:20.601053 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:20.601063 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:20.601082 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:20.601527 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:20.601932 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:20.601937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:20.601940 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:20.602166 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:20.602775 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:20.602782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:20.603297 (+ 515us) client_negotiation.cc:769] Sending connection context
0619 14:56:20.603547 (+ 250us) client_negotiation.cc:240] Negotiation successful
0619 14:56:20.604019 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":395,"threads_started":1}
I20260619 14:56:20.606959 12999 heartbeater.cc:347] Connected to a master server at 127.11.175.254:34471
I20260619 14:56:20.607244 12999 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:20.608006 12999 heartbeater.cc:511] Master 127.11.175.254:34471 requested a full tablet report, sending...
I20260619 14:56:20.609679 12543 ts_manager.cc:194] Registered new tserver with Master: 1274d0d6c5154c49af980b4edcb4f640 (127.11.175.195:35103)
I20260619 14:56:20.610674 12543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:55665
I20260619 14:56:20.613788 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 14:56:20.634848 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.621035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36420 (local address 127.11.175.254:34471)
0619 14:56:20.621293 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:20.621301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:20.621701 (+ 400us) server_negotiation.cc:407] Connection header received
0619 14:56:20.621868 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:20.621873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:20.621943 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:20.622047 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:20.623631 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.624413 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.625780 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.626201 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.626315 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.629131 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:20.629137 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:20.629141 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:20.629180 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:20.631985 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:20.632740 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:20.632745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:20.632747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:20.632883 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:20.633263 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:20.633266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:20.633268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:20.633585 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:20.633804 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:20.634142 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 14:56:20.634338 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 14:56:20.642141 12543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:56:20.649259 12543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:20.678992 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.663981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:35103 (local address 127.0.0.1:43140)
0619 14:56:20.665364 (+ 1383us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:20.665417 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:20.665634 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:20.667084 (+ 1450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:20.667096 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:20.667151 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:20.667713 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.667741 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.670855 (+ 3114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.670864 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.674526 (+ 3662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.674556 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.677559 (+ 3003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.677566 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.677892 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.677988 (+ 96us) client_negotiation.cc:769] Sending connection context
0619 14:56:20.678146 (+ 158us) client_negotiation.cc:240] Negotiation successful
0619 14:56:20.678423 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"thread_start_us":2304,"threads_started":1}
I20260619 14:56:20.680671 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.664787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.11.175.195:35103)
0619 14:56:20.666329 (+ 1542us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:20.666338 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:20.666371 (+ 33us) server_negotiation.cc:407] Connection header received
0619 14:56:20.666471 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:20.666479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:20.666812 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:20.667995 (+ 1183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:20.668346 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.670613 (+ 2267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.674906 (+ 4293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.677224 (+ 2318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.679444 (+ 2220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.679757 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:20.679855 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 14:56:20.680096 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1345,"thread_start_us":1180,"threads_started":1}
I20260619 14:56:20.685340 12934 tablet_service.cc:1467] Processing CreateTablet for tablet 30f48c2354b140db9aa973a71db83aeb (DEFAULT_TABLE table=test-table [id=3fcf2a495f7444929cee730724648ef4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:20.687122 12934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30f48c2354b140db9aa973a71db83aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:20.690886 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.664782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60654 (local address 127.11.175.194:43299)
0619 14:56:20.682164 (+ 17382us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:20.682172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:20.682197 (+ 25us) server_negotiation.cc:407] Connection header received
0619 14:56:20.682297 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:20.682304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:20.682521 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:20.682790 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:20.683738 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.685427 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.687825 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.689365 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.689540 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.689733 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:20.690318 (+ 585us) server_negotiation.cc:299] Negotiation successful
0619 14:56:20.690508 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17190,"thread_start_us":17062,"threads_started":1}
I20260619 14:56:20.690925 13012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.664567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:43299 (local address 127.0.0.1:60654)
0619 14:56:20.666690 (+ 2123us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:20.666715 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:20.666866 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:20.682994 (+ 16128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:20.682998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:20.683014 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:20.683487 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.683502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.685635 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.685641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.687639 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.687657 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.689877 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.689884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.690067 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.690116 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 14:56:20.690229 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 14:56:20.690401 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1917,"mutex_wait_us":30,"thread_start_us":310,"threads_started":1}
I20260619 14:56:20.690966 13016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.666703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:44493 (local address 127.0.0.1:36914)
0619 14:56:20.677730 (+ 11027us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:20.677751 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:20.677888 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:20.682958 (+ 5070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:20.682966 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:20.683001 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:20.683673 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.683687 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.685616 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.685623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.687968 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.687986 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.690144 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.690149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.690328 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.690356 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 14:56:20.690423 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 14:56:20.690551 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10885,"mutex_wait_us":29,"thread_start_us":10721,"threads_started":1}
I20260619 14:56:20.691675 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.667388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36914 (local address 127.11.175.193:44493)
0619 14:56:20.681878 (+ 14490us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:20.681888 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:20.681920 (+ 32us) server_negotiation.cc:407] Connection header received
0619 14:56:20.682032 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:20.682041 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:20.682398 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:20.682622 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:20.683854 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.685427 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.688158 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.689613 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.689824 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.689999 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:20.691084 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0619 14:56:20.691281 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14289,"thread_start_us":14096,"threads_started":1}
I20260619 14:56:20.695326 12801 tablet_service.cc:1467] Processing CreateTablet for tablet 30f48c2354b140db9aa973a71db83aeb (DEFAULT_TABLE table=test-table [id=3fcf2a495f7444929cee730724648ef4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:20.695614 12667 tablet_service.cc:1467] Processing CreateTablet for tablet 30f48c2354b140db9aa973a71db83aeb (DEFAULT_TABLE table=test-table [id=3fcf2a495f7444929cee730724648ef4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:20.696782 12801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30f48c2354b140db9aa973a71db83aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:20.697093 12667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30f48c2354b140db9aa973a71db83aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:20.702396 13018 tablet_bootstrap.cc:492] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640: Bootstrap starting.
I20260619 14:56:20.706501 13018 tablet_bootstrap.cc:654] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:20.708058 13018 log.cc:824] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:20.710704 13020 tablet_bootstrap.cc:492] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd: Bootstrap starting.
I20260619 14:56:20.714020 13018 tablet_bootstrap.cc:492] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640: No bootstrap required, opened a new log
I20260619 14:56:20.714421 13018 ts_tablet_manager.cc:1397] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260619 14:56:20.714677 13020 tablet_bootstrap.cc:654] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:20.715880 13021 tablet_bootstrap.cc:492] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c: Bootstrap starting.
I20260619 14:56:20.717456 13020 log.cc:824] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:20.719691 13021 tablet_bootstrap.cc:654] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:20.720988 13021 log.cc:824] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:20.721323 13020 tablet_bootstrap.cc:492] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd: No bootstrap required, opened a new log
I20260619 14:56:20.721647 13020 ts_tablet_manager.cc:1397] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260619 14:56:20.724298 13021 tablet_bootstrap.cc:492] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c: No bootstrap required, opened a new log
I20260619 14:56:20.724574 13021 ts_tablet_manager.cc:1397] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 14:56:20.724593 13018 raft_consensus.cc:348] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1274d0d6c5154c49af980b4edcb4f640" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 35103 } } peers { permanent_uuid: "51b6667477a0415bbb377fe80ca7b2bd" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44493 } } peers { permanent_uuid: "02935914c52b4149b4b179b934ca7d2c" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43299 } }
I20260619 14:56:20.725021 13018 raft_consensus.cc:374] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:20.725116 13018 raft_consensus.cc:729] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1274d0d6c5154c49af980b4edcb4f640, State: Initialized, Role: FOLLOWER
I20260619 14:56:20.725786 13018 consensus_queue.cc:260] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1274d0d6c5154c49af980b4edcb4f640" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 35103 } } peers { permanent_uuid: "51b6667477a0415bbb377fe80ca7b2bd" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44493 } } peers { permanent_uuid: "02935914c52b4149b4b179b934ca7d2c" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43299 } }
I20260619 14:56:20.726934 12999 heartbeater.cc:503] Master 127.11.175.254:34471 was elected leader, sending a full tablet report...
I20260619 14:56:20.727777 13018 ts_tablet_manager.cc:1428] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260619 14:56:20.733525 13020 raft_consensus.cc:348] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1274d0d6c5154c49af980b4edcb4f640" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 35103 } } peers { permanent_uuid: "51b6667477a0415bbb377fe80ca7b2bd" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44493 } } peers { permanent_uuid: "02935914c52b4149b4b179b934ca7d2c" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43299 } }
I20260619 14:56:20.734028 13020 raft_consensus.cc:374] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:20.734133 13020 raft_consensus.cc:729] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b6667477a0415bbb377fe80ca7b2bd, State: Initialized, Role: FOLLOWER
I20260619 14:56:20.734825 13020 consensus_queue.cc:260] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1274d0d6c5154c49af980b4edcb4f640" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 35103 } } peers { permanent_uuid: "51b6667477a0415bbb377fe80ca7b2bd" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44493 } } peers { permanent_uuid: "02935914c52b4149b4b179b934ca7d2c" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43299 } }
I20260619 14:56:20.735620 13021 raft_consensus.cc:348] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1274d0d6c5154c49af980b4edcb4f640" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 35103 } } peers { permanent_uuid: "51b6667477a0415bbb377fe80ca7b2bd" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44493 } } peers { permanent_uuid: "02935914c52b4149b4b179b934ca7d2c" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43299 } }
I20260619 14:56:20.736076 13021 raft_consensus.cc:374] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:20.736181 13021 raft_consensus.cc:729] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02935914c52b4149b4b179b934ca7d2c, State: Initialized, Role: FOLLOWER
I20260619 14:56:20.736801 13020 ts_tablet_manager.cc:1428] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260619 14:56:20.736929 13021 consensus_queue.cc:260] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1274d0d6c5154c49af980b4edcb4f640" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 35103 } } peers { permanent_uuid: "51b6667477a0415bbb377fe80ca7b2bd" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44493 } } peers { permanent_uuid: "02935914c52b4149b4b179b934ca7d2c" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43299 } }
I20260619 14:56:20.738148 12866 heartbeater.cc:503] Master 127.11.175.254:34471 was elected leader, sending a full tablet report...
I20260619 14:56:20.738816 13021 ts_tablet_manager.cc:1428] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
W20260619 14:56:20.758080 12733 tablet.cc:2369] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 14:56:20.775841 12867 tablet.cc:2369] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 14:56:20.833681 13000 tablet.cc:2369] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:56:20.989845 13025 raft_consensus.cc:482] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:20.990242 13025 raft_consensus.cc:504] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1274d0d6c5154c49af980b4edcb4f640" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 35103 } } peers { permanent_uuid: "51b6667477a0415bbb377fe80ca7b2bd" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44493 } } peers { permanent_uuid: "02935914c52b4149b4b179b934ca7d2c" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43299 } }
I20260619 14:56:20.992106 13025 leader_election.cc:288] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1274d0d6c5154c49af980b4edcb4f640 (127.11.175.195:35103), 02935914c52b4149b4b179b934ca7d2c (127.11.175.194:43299)
I20260619 14:56:20.999331 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.992618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:52779 (local address 127.11.175.194:43299)
0619 14:56:20.992820 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:20.992831 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:20.993487 (+ 656us) server_negotiation.cc:407] Connection header received
0619 14:56:20.993737 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:20.993743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:20.993815 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:20.993919 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:20.994550 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.995380 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.997480 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.998277 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.998474 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.998558 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:20.998903 (+ 345us) server_negotiation.cc:299] Negotiation successful
0619 14:56:20.999039 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 14:56:20.999456 13027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.992414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:43299 (local address 127.11.175.193:52779)
0619 14:56:20.993352 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:20.993368 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:20.993566 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:20.993938 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:20.993942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:20.994031 (+ 89us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:20.994390 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.994403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.995521 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.995527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.997306 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.997321 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.998510 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.998516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.998743 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.998769 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 14:56:20.998879 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 14:56:20.999027 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"spinlock_wait_cycles":25984,"thread_start_us":635,"threads_started":1}
I20260619 14:56:20.999881 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.992513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:58533 (local address 127.11.175.195:35103)
0619 14:56:20.992723 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:20.992732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:20.993912 (+ 1180us) server_negotiation.cc:407] Connection header received
0619 14:56:20.994124 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:20.994128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:20.994187 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:20.994288 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:20.994963 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.995816 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.997890 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.998719 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.998902 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.998983 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:20.999268 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 14:56:20.999402 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 14:56:21.000049 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:20.992560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:35103 (local address 127.11.175.193:58533)
0619 14:56:20.993797 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:20.993815 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:20.993967 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:20.994305 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:20.994313 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:20.994341 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:20.994787 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.994797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.995988 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.995999 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.997709 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:20.997721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:20.998945 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:20.998952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:20.999106 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:20.999130 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 14:56:20.999240 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 14:56:20.999374 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1132,"thread_start_us":914,"threads_started":1}
I20260619 14:56:21.000514 12821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f48c2354b140db9aa973a71db83aeb" candidate_uuid: "51b6667477a0415bbb377fe80ca7b2bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02935914c52b4149b4b179b934ca7d2c" is_pre_election: true
I20260619 14:56:21.001012 12954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f48c2354b140db9aa973a71db83aeb" candidate_uuid: "51b6667477a0415bbb377fe80ca7b2bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1274d0d6c5154c49af980b4edcb4f640" is_pre_election: true
I20260619 14:56:21.001156 12821 raft_consensus.cc:2454] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b6667477a0415bbb377fe80ca7b2bd in term 0.
I20260619 14:56:21.001547 12954 raft_consensus.cc:2454] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b6667477a0415bbb377fe80ca7b2bd in term 0.
I20260619 14:56:21.002039 12623 leader_election.cc:302] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02935914c52b4149b4b179b934ca7d2c, 51b6667477a0415bbb377fe80ca7b2bd; no voters:
I20260619 14:56:21.002534 13025 raft_consensus.cc:2781] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:21.002734 13025 raft_consensus.cc:482] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:21.002862 13025 raft_consensus.cc:3037] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:21.005191 13025 raft_consensus.cc:504] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1274d0d6c5154c49af980b4edcb4f640" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 35103 } } peers { permanent_uuid: "51b6667477a0415bbb377fe80ca7b2bd" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44493 } } peers { permanent_uuid: "02935914c52b4149b4b179b934ca7d2c" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43299 } }
I20260619 14:56:21.005860 13025 leader_election.cc:288] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [CANDIDATE]: Term 1 election: Requested vote from peers 1274d0d6c5154c49af980b4edcb4f640 (127.11.175.195:35103), 02935914c52b4149b4b179b934ca7d2c (127.11.175.194:43299)
I20260619 14:56:21.006423 12954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f48c2354b140db9aa973a71db83aeb" candidate_uuid: "51b6667477a0415bbb377fe80ca7b2bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1274d0d6c5154c49af980b4edcb4f640"
I20260619 14:56:21.006620 12821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f48c2354b140db9aa973a71db83aeb" candidate_uuid: "51b6667477a0415bbb377fe80ca7b2bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02935914c52b4149b4b179b934ca7d2c"
I20260619 14:56:21.006742 12954 raft_consensus.cc:3037] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:21.006933 12821 raft_consensus.cc:3037] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:21.009654 12821 raft_consensus.cc:2454] T 30f48c2354b140db9aa973a71db83aeb P 02935914c52b4149b4b179b934ca7d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b6667477a0415bbb377fe80ca7b2bd in term 1.
I20260619 14:56:21.009945 12954 raft_consensus.cc:2454] T 30f48c2354b140db9aa973a71db83aeb P 1274d0d6c5154c49af980b4edcb4f640 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b6667477a0415bbb377fe80ca7b2bd in term 1.
I20260619 14:56:21.010157 12623 leader_election.cc:302] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02935914c52b4149b4b179b934ca7d2c, 51b6667477a0415bbb377fe80ca7b2bd; no voters:
I20260619 14:56:21.010536 13025 raft_consensus.cc:2781] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:21.011078 13025 raft_consensus.cc:686] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [term 1 LEADER]: Becoming Leader. State: Replica: 51b6667477a0415bbb377fe80ca7b2bd, State: Running, Role: LEADER
I20260619 14:56:21.011627 13025 consensus_queue.cc:237] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1274d0d6c5154c49af980b4edcb4f640" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 35103 } } peers { permanent_uuid: "51b6667477a0415bbb377fe80ca7b2bd" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44493 } } peers { permanent_uuid: "02935914c52b4149b4b179b934ca7d2c" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43299 } }
I20260619 14:56:21.016633 12542 catalog_manager.cc:5495] T 30f48c2354b140db9aa973a71db83aeb P 51b6667477a0415bbb377fe80ca7b2bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b6667477a0415bbb377fe80ca7b2bd (127.11.175.193). New cstate: current_term: 1 leader_uuid: "51b6667477a0415bbb377fe80ca7b2bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1274d0d6c5154c49af980b4edcb4f640" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 35103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b6667477a0415bbb377fe80ca7b2bd" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02935914c52b4149b4b179b934ca7d2c" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43299 } health_report { overall_health: UNKNOWN } } }
Jun 19 14:56:21 dist-test-slave-76rk krb5kdc[12498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:56:21.051383 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:21.035889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36930 (local address 127.11.175.193:44493)
0619 14:56:21.036073 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:21.036077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:21.036587 (+ 510us) server_negotiation.cc:407] Connection header received
0619 14:56:21.036806 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:21.036811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:21.036877 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:21.036983 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:21.038474 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.039264 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.040539 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.041013 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.041161 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:21.045329 (+ 4168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:21.045345 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:21.045371 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:21.045418 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:21.048490 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.049269 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.049277 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.049280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.049429 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.049844 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.049848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.049850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.050220 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:21.050444 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:21.050808 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 14:56:21.051011 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 14:56:21 dist-test-slave-76rk krb5kdc[12498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880981, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 14:56:21 dist-test-slave-76rk krb5kdc[12498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880981, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:56:21.101293 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:21.086303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36934 (local address 127.11.175.193:44493)
0619 14:56:21.086571 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:21.086579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:21.086951 (+ 372us) server_negotiation.cc:407] Connection header received
0619 14:56:21.087090 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:21.087095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:21.087164 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:21.087254 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:21.088710 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.089479 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.090834 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.091243 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.091358 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:21.095488 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:21.095493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:21.095496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:21.095533 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:21.098416 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.099179 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.099183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.099185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.099326 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.099832 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.099837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.099839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.100116 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:21.100358 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:21.100685 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 14:56:21.100905 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
Jun 19 14:56:21 dist-test-slave-76rk krb5kdc[12498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 14:56:21 dist-test-slave-76rk krb5kdc[12498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:56:21.149678 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:21.135067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36936 (local address 127.11.175.193:44493)
0619 14:56:21.135275 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:21.135285 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:21.135668 (+ 383us) server_negotiation.cc:407] Connection header received
0619 14:56:21.135887 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:21.135892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:21.135976 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:21.136088 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:21.137364 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.138127 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.139447 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.139888 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.140022 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:21.144099 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:21.144104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:21.144107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:21.144144 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:21.146946 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.147698 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.147712 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.147718 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.147862 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.148228 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.148233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.148238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.148512 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:21.148799 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:21.149101 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 14:56:21.149326 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 14:56:21.151485 12647 tablet_service.cc:2908] Rejecting scan request for tablet 30f48c2354b140db9aa973a71db83aeb: Uninitialized: clean time has not yet been initialized
I20260619 14:56:21.154590 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 12604
I20260619 14:56:21.168640 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:21.155887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.11.175.193:44493)
0619 14:56:21.156056 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:21.156062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:21.156699 (+ 637us) server_negotiation.cc:407] Connection header received
0619 14:56:21.156858 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:21.156862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:21.156932 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:21.157023 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:21.158305 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.159252 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.160500 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.161004 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.161157 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:21.163479 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:21.163485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:21.163488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:21.163521 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:21.166054 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.166756 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.166760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.166763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.166878 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.167242 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.167245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.167247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.167494 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:21.167670 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:21.168037 (+ 367us) server_negotiation.cc:299] Negotiation successful
0619 14:56:21.168246 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 14:56:21.169387 12727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:21.262986 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 12604
I20260619 14:56:21.280754 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 12738
Jun 19 14:56:21 dist-test-slave-76rk krb5kdc[12498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:56:21.296391 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:21.281602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60666 (local address 127.11.175.194:43299)
0619 14:56:21.281802 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:21.281806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:21.281848 (+ 42us) server_negotiation.cc:407] Connection header received
0619 14:56:21.282057 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:21.282062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:21.282137 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:21.282227 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:21.283560 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.284346 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.285777 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.286189 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.286295 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:21.290398 (+ 4103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:21.290414 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:21.290429 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:21.290465 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:21.293440 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.294152 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.294161 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.294163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.294321 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.294774 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.294778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.294781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.295151 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:21.295410 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:21.295812 (+ 402us) server_negotiation.cc:299] Negotiation successful
0619 14:56:21.296049 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 14:56:21.297050 12861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:21.387439 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 12738
I20260619 14:56:21.404860 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 12871
Jun 19 14:56:21 dist-test-slave-76rk krb5kdc[12498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:56:21.421236 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:21.405721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.11.175.195:35103)
0619 14:56:21.405936 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:21.405940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:21.405959 (+ 19us) server_negotiation.cc:407] Connection header received
0619 14:56:21.406112 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:21.406117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:21.406180 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:21.406262 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:21.407625 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.408361 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.409817 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:21.410208 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:21.410316 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:21.415291 (+ 4975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:21.415308 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:21.415337 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:21.415378 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:21.418219 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.419029 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.419037 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.419039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.419175 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:21.419607 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:21.419610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:21.419612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:21.420022 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:21.420368 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:21.420649 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 14:56:21.420874 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 14:56:21.421911 12994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:21.510715 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 12871
I20260619 14:56:21.528059 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 12511
I20260619 14:56:21.529235 12573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:21.640026 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 12511
2026-06-19T14:56:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4525 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:56:21 dist-test-slave-76rk krb5kdc[13048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:56:21 dist-test-slave-76rk krb5kdc[13048](info): set up 2 sockets
Jun 19 14:56:21 dist-test-slave-76rk krb5kdc[13048](info): commencing operation
krb5kdc: starting...
W20260619 14:56:22.786856 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:56:22 dist-test-slave-76rk krb5kdc[13048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:56:22.995548 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:33759
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:33759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:23.345263 13061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:23.345664 13061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:23.345852 13061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:23.353701 13061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:23.353885 13061 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 14:56:23.353986 13061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:23.354122 13061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:23.354215 13061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:23.354311 13061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:56:23.364929 13061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:33759
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:33759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:23.366214 13061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:23.367620 13061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:23.380434 13069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:23.380468 13067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:23.381212 13066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:23.381785 13061 server_base.cc:1034] running on GCE node
I20260619 14:56:23.382881 13061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:23.384446 13061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:23.385776 13061 hybrid_clock.cc:648] HybridClock initialized: now 1781880983385709 us; error 38 us; skew 500 ppm
I20260619 14:56:23.386222 13061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:23 dist-test-slave-76rk krb5kdc[13048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880983, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:23.390022 13061 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:56:23.391892 13061 webserver.cc:460] Webserver started at http://127.11.175.254:34091/ using document root <none> and password file <none>
I20260619 14:56:23.392673 13061 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:23.392819 13061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:23.393175 13061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:23.396477 13061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "9f197370119c423b8f33b3390efd3e1b"
format_stamp: "Formatted at 2026-06-19 14:56:23 on dist-test-slave-76rk"
server_key: "bdce9979fa35379a16ac15bf80b14ac3"
server_key_iv: "79c6992bf6952aaed2d6c82e4645e485"
server_key_version: "encryptionkey@0"
I20260619 14:56:23.397342 13061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "9f197370119c423b8f33b3390efd3e1b"
format_stamp: "Formatted at 2026-06-19 14:56:23 on dist-test-slave-76rk"
server_key: "bdce9979fa35379a16ac15bf80b14ac3"
server_key_iv: "79c6992bf6952aaed2d6c82e4645e485"
server_key_version: "encryptionkey@0"
I20260619 14:56:23.403461 13061 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260619 14:56:23.407661 13076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:23.409355 13061 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260619 14:56:23.409602 13061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "9f197370119c423b8f33b3390efd3e1b"
format_stamp: "Formatted at 2026-06-19 14:56:23 on dist-test-slave-76rk"
server_key: "bdce9979fa35379a16ac15bf80b14ac3"
server_key_iv: "79c6992bf6952aaed2d6c82e4645e485"
server_key_version: "encryptionkey@0"
I20260619 14:56:23.409888 13061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:23.445912 13061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:23.450884 13061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:23.451213 13061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:23.476476 13061 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:33759
I20260619 14:56:23.476528 13137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:33759 every 8 connection(s)
I20260619 14:56:23.477945 13061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:23.481997 13138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:23.487421 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 13061
I20260619 14:56:23.487680 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:23.488353 11967 external_mini_cluster.cc:1428] Setting key 97e4b353d01f1db03c863f95aa9b60e9
I20260619 14:56:23.494782 13138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b: Bootstrap starting.
I20260619 14:56:23.498878 13138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:23.500028 13138 log.cc:824] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:23.503175 13138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b: No bootstrap required, opened a new log
Jun 19 14:56:23 dist-test-slave-76rk krb5kdc[13048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:23.511451 13138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f197370119c423b8f33b3390efd3e1b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33759 } }
I20260619 14:56:23.511876 13138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:23.512022 13138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f197370119c423b8f33b3390efd3e1b, State: Initialized, Role: FOLLOWER
I20260619 14:56:23.512696 13138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f197370119c423b8f33b3390efd3e1b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33759 } }
I20260619 14:56:23.513000 13138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:23.513156 13138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:23.513360 13138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:23.513652 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:23.490484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53626 (local address 127.11.175.254:33759)
0619 14:56:23.491496 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:23.491512 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:23.491568 (+ 56us) server_negotiation.cc:407] Connection header received
0619 14:56:23.494026 (+ 2458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:23.494057 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:23.494625 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:23.495151 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:23.496561 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:23.498565 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:23.499997 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:23.501047 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:23.501295 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:23.505596 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:23.505604 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:23.505621 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:23.505674 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:23.508988 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:23.509774 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:23.509783 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:23.509790 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:23.509976 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:23.510395 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:23.510402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:23.510410 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:23.511070 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:23.511386 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:23.511849 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 14:56:23.512376 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":409,"threads_started":1}
I20260619 14:56:23.516433 13138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f197370119c423b8f33b3390efd3e1b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33759 } }
I20260619 14:56:23.517068 13138 leader_election.cc:302] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f197370119c423b8f33b3390efd3e1b; no voters:
I20260619 14:56:23.517773 13138 leader_election.cc:288] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:23.518011 13143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:23.519011 13143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [term 1 LEADER]: Becoming Leader. State: Replica: 9f197370119c423b8f33b3390efd3e1b, State: Running, Role: LEADER
I20260619 14:56:23.519899 13138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:23.520509 13143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f197370119c423b8f33b3390efd3e1b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33759 } }
I20260619 14:56:23.523597 13144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f197370119c423b8f33b3390efd3e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f197370119c423b8f33b3390efd3e1b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33759 } } }
I20260619 14:56:23.524096 13144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:23.524266 13145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f197370119c423b8f33b3390efd3e1b. Latest consensus state: current_term: 1 leader_uuid: "9f197370119c423b8f33b3390efd3e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f197370119c423b8f33b3390efd3e1b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33759 } } }
I20260619 14:56:23.524590 13145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:23.525179 13154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:23.531843 13154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:23.541417 13154 catalog_manager.cc:1347] Generated new cluster ID: 617aab96a3f546c59f48f299ce46cd96
I20260619 14:56:23.541566 13154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:23.555279 13154 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:23.556923 13154 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:23.566027 13154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b: Generated new TSK 0
I20260619 14:56:23.567289 13154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:56:23.623937 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:33759
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:23.959201 13164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:23.959492 13164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:23.959592 13164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:23.967013 13164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:23.967131 13164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:23.967222 13164 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 14:56:23.967306 13164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:56:23.977284 13164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.175.254:33759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:23.978515 13164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:23.979885 13164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:23.994051 13172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:23.993970 13169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:23.994014 13170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:23.994905 13164 server_base.cc:1034] running on GCE node
I20260619 14:56:23.995517 13164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:23.996505 13164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:23.997753 13164 hybrid_clock.cc:648] HybridClock initialized: now 1781880983997683 us; error 43 us; skew 500 ppm
I20260619 14:56:23.998036 13164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:23 dist-test-slave-76rk krb5kdc[13048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880983, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:24.001513 13164 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:56:24.003141 13164 webserver.cc:460] Webserver started at http://127.11.175.193:35443/ using document root <none> and password file <none>
I20260619 14:56:24.003893 13164 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:24.004027 13164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:24.004362 13164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:24.007476 13164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "3d28479f2ed04696991b13fcd4640dc1"
format_stamp: "Formatted at 2026-06-19 14:56:24 on dist-test-slave-76rk"
server_key: "a551c1c66f62f6a8b929c9707bf40613"
server_key_iv: "b8b1a1c07fac25c7002b50b479b6e96d"
server_key_version: "encryptionkey@0"
I20260619 14:56:24.008317 13164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "3d28479f2ed04696991b13fcd4640dc1"
format_stamp: "Formatted at 2026-06-19 14:56:24 on dist-test-slave-76rk"
server_key: "a551c1c66f62f6a8b929c9707bf40613"
server_key_iv: "b8b1a1c07fac25c7002b50b479b6e96d"
server_key_version: "encryptionkey@0"
I20260619 14:56:24.014092 13164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 14:56:24.017829 13179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:24.019385 13164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:56:24.019570 13164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "3d28479f2ed04696991b13fcd4640dc1"
format_stamp: "Formatted at 2026-06-19 14:56:24 on dist-test-slave-76rk"
server_key: "a551c1c66f62f6a8b929c9707bf40613"
server_key_iv: "b8b1a1c07fac25c7002b50b479b6e96d"
server_key_version: "encryptionkey@0"
I20260619 14:56:24.019821 13164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:24.044838 13164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:24.049415 13164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:24.049712 13164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:24.052623 13164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:24.052744 13164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:24.052893 13164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:24.052968 13164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 14:56:24 dist-test-slave-76rk krb5kdc[13048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880983, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:24.078510 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:24.053462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:37123 (local address 127.11.175.254:33759)
0619 14:56:24.053928 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:24.053932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:24.055391 (+ 1459us) server_negotiation.cc:407] Connection header received
0619 14:56:24.057559 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:24.057566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:24.057659 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:24.057794 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:24.060736 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.061775 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.065259 (+ 3484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.065823 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.066011 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:24.071533 (+ 5522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:24.071538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:24.071542 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:24.071580 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:24.074622 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:24.075884 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:24.075891 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:24.075896 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:24.076099 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:24.076643 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:24.076648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:24.076659 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:24.076947 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:24.077119 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:24.077884 (+ 765us) server_negotiation.cc:299] Negotiation successful
0619 14:56:24.078126 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":249,"threads_started":1}
I20260619 14:56:24.079607 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:24.054235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:33759 (local address 127.11.175.193:37123)
0619 14:56:24.055206 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:24.055247 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:24.057198 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:24.058132 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:24.058150 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:24.059185 (+ 1035us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:24.060421 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:24.060469 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.061988 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.062001 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:24.065034 (+ 3033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:24.065062 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.066138 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.066147 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:24.066442 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:24.067287 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:24.067345 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:24.071217 (+ 3872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:24.074851 (+ 3634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:24.074862 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:24.074890 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:24.075672 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:24.076251 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:24.076257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:24.076261 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:24.076497 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:24.077089 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:24.077097 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:24.077499 (+ 402us) client_negotiation.cc:769] Sending connection context
0619 14:56:24.077776 (+ 277us) client_negotiation.cc:240] Negotiation successful
0619 14:56:24.078358 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":461,"threads_started":1}
I20260619 14:56:24.104630 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:24.086111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:34435 (local address 127.11.175.254:33759)
0619 14:56:24.086355 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:24.086365 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:24.087435 (+ 1070us) server_negotiation.cc:407] Connection header received
0619 14:56:24.087672 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:24.087680 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:24.087981 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:24.088114 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:24.089978 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.090848 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.093157 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.093808 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.093968 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:24.096576 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:24.096581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:24.096584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:24.096619 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:24.100819 (+ 4200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:24.101852 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:24.101861 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:24.101866 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:24.102065 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:24.102613 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:24.102623 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:24.102630 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:24.103181 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:24.103401 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:24.103992 (+ 591us) server_negotiation.cc:299] Negotiation successful
0619 14:56:24.104244 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 14:56:24.104843 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:24.086082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:33759 (local address 127.11.175.193:34435)
0619 14:56:24.087318 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:24.087338 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:24.087514 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:24.088172 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:24.088180 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:24.089186 (+ 1006us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:24.089737 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:24.089759 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.091017 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.091023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:24.092979 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:24.092993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.094019 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.094025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:24.094189 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:24.094993 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:24.095015 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:24.096330 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:24.101002 (+ 4672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:24.101012 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:24.101018 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:24.101622 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:24.102212 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:24.102217 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:24.102219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:24.102447 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:24.103341 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:24.103351 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:24.103692 (+ 341us) client_negotiation.cc:769] Sending connection context
0619 14:56:24.103893 (+ 201us) client_negotiation.cc:240] Negotiation successful
0619 14:56:24.104296 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1076,"thread_start_us":719,"threads_started":1}
I20260619 14:56:24.110028 13164 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:35217
I20260619 14:56:24.110036 13299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:35217 every 8 connection(s)
I20260619 14:56:24.111305 13164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:56:24.113245 13300 heartbeater.cc:347] Connected to a master server at 127.11.175.254:33759
I20260619 14:56:24.113476 13300 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:24.114095 13300 heartbeater.cc:511] Master 127.11.175.254:33759 requested a full tablet report, sending...
I20260619 14:56:24.115140 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 13164
I20260619 14:56:24.115403 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:56:24.116035 11967 external_mini_cluster.cc:1428] Setting key 8f7bebec4548dc829303e35a51de2c39
I20260619 14:56:24.116416 13093 ts_manager.cc:194] Registered new tserver with Master: 3d28479f2ed04696991b13fcd4640dc1 (127.11.175.193:35217)
I20260619 14:56:24.118512 13093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:37123
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:56:24.160069 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:33759
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:24.493824 13304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:24.494109 13304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:24.494264 13304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:24.501734 13304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:24.501852 13304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:24.502035 13304 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 14:56:24.502144 13304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:56:24.512257 13304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.175.254:33759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:24.513689 13304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:24.515151 13304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:24.529476 13310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:24.529556 13312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:24.529556 13309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:24.530128 13304 server_base.cc:1034] running on GCE node
I20260619 14:56:24.530879 13304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:24.531797 13304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:24.533010 13304 hybrid_clock.cc:648] HybridClock initialized: now 1781880984532948 us; error 32 us; skew 500 ppm
I20260619 14:56:24.533305 13304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:24 dist-test-slave-76rk krb5kdc[13048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880984, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:24.536707 13304 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:56:24.538183 13304 webserver.cc:460] Webserver started at http://127.11.175.194:33265/ using document root <none> and password file <none>
I20260619 14:56:24.538998 13304 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:24.539129 13304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:24.539443 13304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:24.542400 13304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "55be6f806e1f48b8a69e2cf4bf6fe251"
format_stamp: "Formatted at 2026-06-19 14:56:24 on dist-test-slave-76rk"
server_key: "ad899456d1c69114dfb772f098ff1637"
server_key_iv: "601185493038c1442a4b31f3b6c595f4"
server_key_version: "encryptionkey@0"
I20260619 14:56:24.543303 13304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "55be6f806e1f48b8a69e2cf4bf6fe251"
format_stamp: "Formatted at 2026-06-19 14:56:24 on dist-test-slave-76rk"
server_key: "ad899456d1c69114dfb772f098ff1637"
server_key_iv: "601185493038c1442a4b31f3b6c595f4"
server_key_version: "encryptionkey@0"
I20260619 14:56:24.548560 13304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:56:24.552253 13319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:24.553743 13304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 14:56:24.553964 13304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "55be6f806e1f48b8a69e2cf4bf6fe251"
format_stamp: "Formatted at 2026-06-19 14:56:24 on dist-test-slave-76rk"
server_key: "ad899456d1c69114dfb772f098ff1637"
server_key_iv: "601185493038c1442a4b31f3b6c595f4"
server_key_version: "encryptionkey@0"
I20260619 14:56:24.554210 13304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:24.569936 13304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:24.574441 13304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:24.574803 13304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:24.577750 13304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:24.577868 13304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:24.578054 13304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:24.578141 13304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 14:56:24 dist-test-slave-76rk krb5kdc[13048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880984, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:24.601761 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:24.578106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:54573 (local address 127.11.175.254:33759)
0619 14:56:24.578335 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:24.578343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:24.579885 (+ 1542us) server_negotiation.cc:407] Connection header received
0619 14:56:24.582992 (+ 3107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:24.582999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:24.583097 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:24.583220 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:24.586001 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.587094 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.589306 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.589792 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.589938 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:24.594697 (+ 4759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:24.594706 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:24.594713 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:24.594764 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:24.598178 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:24.599192 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:24.599202 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:24.599208 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:24.599391 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:24.599974 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:24.599980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:24.599984 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:24.600282 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:24.600456 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:24.601174 (+ 718us) server_negotiation.cc:299] Negotiation successful
0619 14:56:24.601402 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 14:56:24.603068 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:24.578802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:33759 (local address 127.11.175.194:54573)
0619 14:56:24.579683 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:24.579734 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:24.582564 (+ 2830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:24.583704 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:24.583721 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:24.584798 (+ 1077us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:24.585765 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:24.585811 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.587279 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.587289 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:24.589143 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:24.589154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.590047 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.590057 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:24.590307 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:24.591114 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:24.591138 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:24.594412 (+ 3274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:24.598466 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:24.598472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:24.598485 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:24.599037 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:24.599536 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:24.599542 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:24.599547 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:24.599845 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:24.600429 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:24.600438 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:24.600811 (+ 373us) client_negotiation.cc:769] Sending connection context
0619 14:56:24.601077 (+ 266us) client_negotiation.cc:240] Negotiation successful
0619 14:56:24.601625 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":357,"threads_started":1}
I20260619 14:56:24.626247 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:24.609366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:33759 (local address 127.11.175.194:49185)
0619 14:56:24.610111 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:24.610136 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:24.610325 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:24.610724 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:24.610731 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:24.611664 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:24.612232 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:24.612252 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.613558 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.613569 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:24.615836 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:24.615858 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.617122 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.617134 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:24.617371 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:24.618107 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:24.618124 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:24.619044 (+ 920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:24.623478 (+ 4434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:24.623482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:24.623484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:24.623931 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:24.624446 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:24.624449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:24.624451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:24.624563 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:24.625387 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:24.625391 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:24.625544 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 14:56:24.625654 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 14:56:24.625917 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":365,"threads_started":1}
I20260619 14:56:24.626828 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:24.609296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:49185 (local address 127.11.175.254:33759)
0619 14:56:24.609500 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:24.609508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:24.610249 (+ 741us) server_negotiation.cc:407] Connection header received
0619 14:56:24.610475 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:24.610478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:24.610553 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:24.610676 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:24.612444 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.613408 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.616099 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:24.616873 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:24.617036 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:24.619215 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:24.619223 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:24.619230 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:24.619284 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:24.623335 (+ 4051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:24.624061 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:24.624070 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:24.624076 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:24.624332 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:24.624680 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:24.624688 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:24.624693 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:24.625246 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:24.625451 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:24.625821 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 14:56:24.626136 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 14:56:24.634055 13304 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:38631
I20260619 14:56:24.634085 13438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:38631 every 8 connection(s)
I20260619 14:56:24.635365 13304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:56:24.637462 13439 heartbeater.cc:347] Connected to a master server at 127.11.175.254:33759
I20260619 14:56:24.637684 13439 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:24.638350 13439 heartbeater.cc:511] Master 127.11.175.254:33759 requested a full tablet report, sending...
I20260619 14:56:24.639426 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 13304
I20260619 14:56:24.639671 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:56:24.639951 13093 ts_manager.cc:194] Registered new tserver with Master: 55be6f806e1f48b8a69e2cf4bf6fe251 (127.11.175.194:38631)
I20260619 14:56:24.640134 11967 external_mini_cluster.cc:1428] Setting key 87a3be7cfbecbb3ef59d58dab2d53c1d
I20260619 14:56:24.641052 13093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:54573
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:56:24.681696 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:33759
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:25.015381 13443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:25.015738 13443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:25.015918 13443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:25.024082 13443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:25.024247 13443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:25.024405 13443 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 14:56:25.024542 13443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:56:25.034765 13443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.175.254:33759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:25.036125 13443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:25.037577 13443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:25.052178 13449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:25.052300 13448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:25.052297 13451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:25.052990 13443 server_base.cc:1034] running on GCE node
I20260619 14:56:25.053670 13443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:25.054739 13443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:25.055990 13443 hybrid_clock.cc:648] HybridClock initialized: now 1781880985055895 us; error 64 us; skew 500 ppm
I20260619 14:56:25.056308 13443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:25 dist-test-slave-76rk krb5kdc[13048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880985, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:25.059854 13443 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:56:25.061602 13443 webserver.cc:460] Webserver started at http://127.11.175.195:37725/ using document root <none> and password file <none>
I20260619 14:56:25.062314 13443 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:25.062444 13443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:25.062834 13443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:25.065696 13443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "24fd0efba8a04c198d1ea5aeda73e95d"
format_stamp: "Formatted at 2026-06-19 14:56:25 on dist-test-slave-76rk"
server_key: "8e0d63c77aecba47a2d5819b34412166"
server_key_iv: "3183f09ee4b3545aa0af5f0d9e77da83"
server_key_version: "encryptionkey@0"
I20260619 14:56:25.066506 13443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "24fd0efba8a04c198d1ea5aeda73e95d"
format_stamp: "Formatted at 2026-06-19 14:56:25 on dist-test-slave-76rk"
server_key: "8e0d63c77aecba47a2d5819b34412166"
server_key_iv: "3183f09ee4b3545aa0af5f0d9e77da83"
server_key_version: "encryptionkey@0"
I20260619 14:56:25.071836 13443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 14:56:25.075645 13458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:25.076982 13443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:25.077171 13443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "24fd0efba8a04c198d1ea5aeda73e95d"
format_stamp: "Formatted at 2026-06-19 14:56:25 on dist-test-slave-76rk"
server_key: "8e0d63c77aecba47a2d5819b34412166"
server_key_iv: "3183f09ee4b3545aa0af5f0d9e77da83"
server_key_version: "encryptionkey@0"
I20260619 14:56:25.077404 13443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:25.097709 13443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:25.102272 13443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:25.102509 13443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:25.105233 13443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:25.105330 13443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:25.105484 13443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:25.105567 13443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:25.121891 13300 heartbeater.cc:503] Master 127.11.175.254:33759 was elected leader, sending a full tablet report...
Jun 19 14:56:25 dist-test-slave-76rk krb5kdc[13048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880985, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:25.135118 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.106287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:57267 (local address 127.11.175.254:33759)
0619 14:56:25.106474 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.106478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.108041 (+ 1563us) server_negotiation.cc:407] Connection header received
0619 14:56:25.110696 (+ 2655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.110703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.110808 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.110889 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:25.113489 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.114915 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.118264 (+ 3349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.118975 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.119134 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.125493 (+ 6359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:25.125503 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:25.125512 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:25.125568 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:25.130829 (+ 5261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:25.131935 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:25.131945 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:25.131951 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:25.132166 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:25.132757 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:25.132765 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:25.132771 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:25.133261 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:25.133441 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.134261 (+ 820us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.134592 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 14:56:25.135991 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.106990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:33759 (local address 127.11.175.195:57267)
0619 14:56:25.107866 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:25.107906 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:25.110277 (+ 2371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:25.111463 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:25.111476 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:25.112299 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:25.113241 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.113281 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.115171 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.115181 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.118057 (+ 2876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.118082 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.119322 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.119333 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.119754 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.120752 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:25.120840 (+ 88us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:25.125127 (+ 4287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:25.131107 (+ 5980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:25.131125 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:25.131142 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:25.131773 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:25.132319 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:25.132323 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:25.132326 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:25.132626 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:25.133438 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:25.133445 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:25.133846 (+ 401us) client_negotiation.cc:769] Sending connection context
0619 14:56:25.134100 (+ 254us) client_negotiation.cc:240] Negotiation successful
0619 14:56:25.134515 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":374,"threads_started":1}
I20260619 14:56:25.158708 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.141633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:55637 (local address 127.11.175.254:33759)
0619 14:56:25.141875 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.141884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.142401 (+ 517us) server_negotiation.cc:407] Connection header received
0619 14:56:25.142557 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.142565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.142685 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.142826 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:25.144280 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.145253 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.148087 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.148728 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.148929 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.151618 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:25.151623 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:25.151626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:25.151659 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:25.155470 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:25.156469 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:25.156475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:25.156479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:25.156622 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:25.157115 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:25.157122 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:25.157128 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:25.157465 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:25.157633 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.158073 (+ 440us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.158308 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 14:56:25.158891 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.141612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:33759 (local address 127.11.175.195:55637)
0619 14:56:25.142299 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:25.142321 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:25.142435 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:25.142920 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:25.142924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:25.143454 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:25.144114 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.144129 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.145452 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.145465 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.147907 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.147928 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.149302 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.149319 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.149587 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.150359 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:25.150377 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:25.151403 (+ 1026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:25.155602 (+ 4199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:25.155608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:25.155613 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:25.156329 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:25.156737 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:25.156743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:25.156746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:25.156935 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:25.157583 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:25.157587 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:25.157787 (+ 200us) client_negotiation.cc:769] Sending connection context
0619 14:56:25.157966 (+ 179us) client_negotiation.cc:240] Negotiation successful
0619 14:56:25.158325 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":357,"threads_started":1}
I20260619 14:56:25.160279 13443 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:38449
I20260619 14:56:25.160300 13577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:38449 every 8 connection(s)
I20260619 14:56:25.161535 13443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:56:25.162230 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 13443
I20260619 14:56:25.162437 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:56:25.163034 11967 external_mini_cluster.cc:1428] Setting key a42749ed50c6906d88ffabb11e6b0b4c
I20260619 14:56:25.164297 13578 heartbeater.cc:347] Connected to a master server at 127.11.175.254:33759
I20260619 14:56:25.164741 13578 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:25.165644 13578 heartbeater.cc:511] Master 127.11.175.254:33759 requested a full tablet report, sending...
I20260619 14:56:25.167088 13093 ts_manager.cc:194] Registered new tserver with Master: 24fd0efba8a04c198d1ea5aeda73e95d (127.11.175.195:38449)
I20260619 14:56:25.167917 13093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:57267
I20260619 14:56:25.179088 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 14:56:25 dist-test-slave-76rk krb5kdc[13048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880985, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 14:56:25 dist-test-slave-76rk krb5kdc[13048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880985, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:25.228137 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.213406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.11.175.254:33759)
0619 14:56:25.213686 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.213693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.213956 (+ 263us) server_negotiation.cc:407] Connection header received
0619 14:56:25.214103 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.214107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.214174 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.214260 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:25.215769 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.216500 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.217844 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.218259 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.218357 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.222563 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:25.222572 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:25.222579 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:25.222667 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:25.225501 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:25.226147 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:25.226159 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:25.226162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:25.226288 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:25.226676 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:25.226682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:25.226691 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:25.226998 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:25.227267 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.227576 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.227810 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260619 14:56:25.232463 13093 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:56:25.239468 13093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:25.265678 13594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.254410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:38631 (local address 127.0.0.1:36392)
0619 14:56:25.255400 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:25.255421 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:25.255548 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:25.256194 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:25.256201 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:25.256242 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:25.256800 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.256814 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.259148 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.259156 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.262024 (+ 2868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.262039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.264041 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.264049 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.264342 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.264406 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 14:56:25.264532 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 14:56:25.264743 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":79,"thread_start_us":928,"threads_started":1}
I20260619 14:56:25.265949 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.256080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56578 (local address 127.11.175.195:38449)
0619 14:56:25.256828 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.256838 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.256873 (+ 35us) server_negotiation.cc:407] Connection header received
0619 14:56:25.256968 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.256977 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.257193 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.258168 (+ 975us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:25.258492 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.260000 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.262704 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.264300 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.265130 (+ 830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.265314 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.265397 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.265601 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":1058,"threads_started":1}
I20260619 14:56:25.265666 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.255570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38449 (local address 127.0.0.1:56578)
0619 14:56:25.256112 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:25.256130 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:25.256221 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:25.257407 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:25.257411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:25.257437 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:25.258207 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.258221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.260166 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.260183 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.262498 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.262512 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.264567 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.264573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.264772 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.264809 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 14:56:25.264883 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 14:56:25.265042 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":46,"thread_start_us":658,"threads_started":1}
I20260619 14:56:25.266589 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.254441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36392 (local address 127.11.175.194:38631)
0619 14:56:25.255020 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.255030 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.255567 (+ 537us) server_negotiation.cc:407] Connection header received
0619 14:56:25.255681 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.255689 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.255948 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.257032 (+ 1084us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:25.257283 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.259023 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.262221 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.263643 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.265821 (+ 2178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.265997 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.266077 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.266266 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":294,"threads_started":1}
I20260619 14:56:25.266852 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.253914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.11.175.193:35217)
0619 14:56:25.254573 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.254585 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.255285 (+ 700us) server_negotiation.cc:407] Connection header received
0619 14:56:25.255639 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.255650 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.255914 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.256086 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:25.257504 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.259124 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.263312 (+ 4188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.264850 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.265087 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.265314 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.266055 (+ 741us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.266277 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":324,"threads_started":1}
I20260619 14:56:25.267752 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.253822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:35217 (local address 127.0.0.1:44206)
0619 14:56:25.255100 (+ 1278us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:25.255139 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:25.255359 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:25.256378 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:25.256382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:25.256413 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:25.257197 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.257212 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.259271 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.259278 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.263110 (+ 3832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.263127 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.265524 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.265531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.265779 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.265818 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 14:56:25.267055 (+ 1237us) client_negotiation.cc:240] Negotiation successful
0619 14:56:25.267199 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"mutex_wait_us":285,"thread_start_us":907,"threads_started":1}
I20260619 14:56:25.270310 13369 tablet_service.cc:1467] Processing CreateTablet for tablet 96e922f27c0c433b89f9dc3627f56670 (DEFAULT_TABLE table=test-table [id=95ad3d640cc24c7c955ef9fe0ff70e42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:25.270771 13508 tablet_service.cc:1467] Processing CreateTablet for tablet 96e922f27c0c433b89f9dc3627f56670 (DEFAULT_TABLE table=test-table [id=95ad3d640cc24c7c955ef9fe0ff70e42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:25.271728 13369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e922f27c0c433b89f9dc3627f56670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:25.271489 13230 tablet_service.cc:1467] Processing CreateTablet for tablet 96e922f27c0c433b89f9dc3627f56670 (DEFAULT_TABLE table=test-table [id=95ad3d640cc24c7c955ef9fe0ff70e42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:25.272436 13508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e922f27c0c433b89f9dc3627f56670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:25.272799 13230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e922f27c0c433b89f9dc3627f56670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:25.284188 13597 tablet_bootstrap.cc:492] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: Bootstrap starting.
I20260619 14:56:25.287933 13597 tablet_bootstrap.cc:654] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:25.289346 13599 tablet_bootstrap.cc:492] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: Bootstrap starting.
I20260619 14:56:25.289588 13597 log.cc:824] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:25.291599 13598 tablet_bootstrap.cc:492] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: Bootstrap starting.
I20260619 14:56:25.293409 13599 tablet_bootstrap.cc:654] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:25.295372 13599 log.cc:824] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:25.295975 13598 tablet_bootstrap.cc:654] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:25.296638 13597 tablet_bootstrap.cc:492] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: No bootstrap required, opened a new log
I20260619 14:56:25.296953 13597 ts_tablet_manager.cc:1397] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260619 14:56:25.297313 13598 log.cc:824] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:25.300083 13599 tablet_bootstrap.cc:492] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: No bootstrap required, opened a new log
I20260619 14:56:25.300422 13599 ts_tablet_manager.cc:1397] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260619 14:56:25.303095 13598 tablet_bootstrap.cc:492] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: No bootstrap required, opened a new log
I20260619 14:56:25.303391 13598 ts_tablet_manager.cc:1397] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.009s
I20260619 14:56:25.305337 13597 raft_consensus.cc:348] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } }
I20260619 14:56:25.305732 13597 raft_consensus.cc:374] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:25.305841 13597 raft_consensus.cc:729] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55be6f806e1f48b8a69e2cf4bf6fe251, State: Initialized, Role: FOLLOWER
I20260619 14:56:25.306478 13597 consensus_queue.cc:260] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } }
I20260619 14:56:25.309536 13439 heartbeater.cc:503] Master 127.11.175.254:33759 was elected leader, sending a full tablet report...
I20260619 14:56:25.310166 13597 ts_tablet_manager.cc:1428] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260619 14:56:25.311424 13599 raft_consensus.cc:348] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } }
I20260619 14:56:25.311913 13599 raft_consensus.cc:374] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:25.312016 13599 raft_consensus.cc:729] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24fd0efba8a04c198d1ea5aeda73e95d, State: Initialized, Role: FOLLOWER
I20260619 14:56:25.312661 13599 consensus_queue.cc:260] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } }
I20260619 14:56:25.313886 13578 heartbeater.cc:503] Master 127.11.175.254:33759 was elected leader, sending a full tablet report...
I20260619 14:56:25.314579 13599 ts_tablet_manager.cc:1428] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260619 14:56:25.318027 13598 raft_consensus.cc:348] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } }
I20260619 14:56:25.318547 13598 raft_consensus.cc:374] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:25.318763 13598 raft_consensus.cc:729] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d28479f2ed04696991b13fcd4640dc1, State: Initialized, Role: FOLLOWER
I20260619 14:56:25.319449 13598 consensus_queue.cc:260] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } }
I20260619 14:56:25.321471 13598 ts_tablet_manager.cc:1428] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: Time spent starting tablet: real 0.018s user 0.013s sys 0.001s
I20260619 14:56:25.343998 13603 raft_consensus.cc:482] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:25.344512 13603 raft_consensus.cc:504] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } }
I20260619 14:56:25.346223 13603 leader_election.cc:288] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d28479f2ed04696991b13fcd4640dc1 (127.11.175.193:35217), 24fd0efba8a04c198d1ea5aeda73e95d (127.11.175.195:38449)
I20260619 14:56:25.353709 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.346847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:45837 (local address 127.11.175.193:35217)
0619 14:56:25.347040 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.347045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.347411 (+ 366us) server_negotiation.cc:407] Connection header received
0619 14:56:25.347629 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.347636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.347720 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.347832 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:25.348626 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.349662 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.351714 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.352660 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.352790 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.352844 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.353288 (+ 444us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.353436 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 14:56:25.354067 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.347682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:35369 (local address 127.11.175.195:38449)
0619 14:56:25.347920 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.347926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.348163 (+ 237us) server_negotiation.cc:407] Connection header received
0619 14:56:25.348326 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.348332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.348416 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.348522 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:25.349272 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.350192 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.352087 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.352956 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.353110 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.353183 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.353641 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.353823 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 14:56:25.353806 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.346548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:35217 (local address 127.11.175.194:45837)
0619 14:56:25.347297 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:25.347317 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:25.347460 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:25.348003 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:25.348007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:25.348029 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:25.348435 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.348451 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.349798 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.349803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.351534 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.351549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.352886 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.352893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.353078 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.353117 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 14:56:25.353228 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 14:56:25.353339 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":22,"thread_start_us":419,"threads_started":1}
I20260619 14:56:25.354529 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.347403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38449 (local address 127.11.175.194:35369)
0619 14:56:25.348056 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:25.348069 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:25.348193 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:25.348616 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:25.348621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:25.348649 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:25.349065 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.349077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.350340 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.350343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.351929 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.351942 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.353263 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.353271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.353463 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.353495 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 14:56:25.353596 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 14:56:25.353733 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":21,"thread_start_us":369,"threads_started":1}
I20260619 14:56:25.355510 13255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e922f27c0c433b89f9dc3627f56670" candidate_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d28479f2ed04696991b13fcd4640dc1" is_pre_election: true
I20260619 14:56:25.355571 13530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e922f27c0c433b89f9dc3627f56670" candidate_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" is_pre_election: true
I20260619 14:56:25.356074 13530 raft_consensus.cc:2454] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55be6f806e1f48b8a69e2cf4bf6fe251 in term 0.
I20260619 14:56:25.356063 13255 raft_consensus.cc:2454] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55be6f806e1f48b8a69e2cf4bf6fe251 in term 0.
I20260619 14:56:25.356729 13322 leader_election.cc:302] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d28479f2ed04696991b13fcd4640dc1, 55be6f806e1f48b8a69e2cf4bf6fe251; no voters:
I20260619 14:56:25.357180 13603 raft_consensus.cc:2781] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:25.357363 13603 raft_consensus.cc:482] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:25.357477 13603 raft_consensus.cc:3037] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:25.360148 13603 raft_consensus.cc:504] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } }
I20260619 14:56:25.360723 13603 leader_election.cc:288] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [CANDIDATE]: Term 1 election: Requested vote from peers 3d28479f2ed04696991b13fcd4640dc1 (127.11.175.193:35217), 24fd0efba8a04c198d1ea5aeda73e95d (127.11.175.195:38449)
I20260619 14:56:25.361259 13255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e922f27c0c433b89f9dc3627f56670" candidate_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d28479f2ed04696991b13fcd4640dc1"
I20260619 14:56:25.361310 13530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e922f27c0c433b89f9dc3627f56670" candidate_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24fd0efba8a04c198d1ea5aeda73e95d"
I20260619 14:56:25.361457 13255 raft_consensus.cc:3037] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:25.361608 13530 raft_consensus.cc:3037] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:25.363973 13530 raft_consensus.cc:2454] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55be6f806e1f48b8a69e2cf4bf6fe251 in term 1.
W20260619 14:56:25.363938 13301 tablet.cc:2369] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:56:25.364056 13255 raft_consensus.cc:2454] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55be6f806e1f48b8a69e2cf4bf6fe251 in term 1.
I20260619 14:56:25.364367 13322 leader_election.cc:302] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24fd0efba8a04c198d1ea5aeda73e95d, 55be6f806e1f48b8a69e2cf4bf6fe251; no voters:
I20260619 14:56:25.364670 13603 raft_consensus.cc:2781] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:25.365136 13603 raft_consensus.cc:686] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 LEADER]: Becoming Leader. State: Replica: 55be6f806e1f48b8a69e2cf4bf6fe251, State: Running, Role: LEADER
I20260619 14:56:25.365595 13603 consensus_queue.cc:237] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } }
I20260619 14:56:25.370703 13092 catalog_manager.cc:5495] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55be6f806e1f48b8a69e2cf4bf6fe251 (127.11.175.194). New cstate: current_term: 1 leader_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } health_report { overall_health: HEALTHY } } }
W20260619 14:56:25.387341 13440 tablet.cc:2369] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:56:25.402729 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.396311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.11.175.194:38631)
0619 14:56:25.396509 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.396513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.396647 (+ 134us) server_negotiation.cc:407] Connection header received
0619 14:56:25.396932 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.396936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.397018 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.397149 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:25.397588 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.398379 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.400373 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.400814 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.400920 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.401493 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:25.402086 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:25.402195 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.402370 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.402441 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 14:56:25.411026 13530 raft_consensus.cc:1261] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d [term 1 FOLLOWER]: Refusing update from remote peer 55be6f806e1f48b8a69e2cf4bf6fe251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:25.411167 13255 raft_consensus.cc:1261] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 FOLLOWER]: Refusing update from remote peer 55be6f806e1f48b8a69e2cf4bf6fe251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:25.412032 13608 consensus_queue.cc:1035] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:25.412436 13603 consensus_queue.cc:1035] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 14:56:25.415282 13579 tablet.cc:2369] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:56:25.424681 13611 mvcc.cc:205] Tried to move back new op lower bound from 7298584516233240576 to 7298584516061515776. Current Snapshot: MvccSnapshot[applied={T|T < 7298584516233240576}]
I20260619 14:56:25.425918 13613 mvcc.cc:205] Tried to move back new op lower bound from 7298584516233240576 to 7298584516061515776. Current Snapshot: MvccSnapshot[applied={T|T < 7298584516233240576}]
I20260619 14:56:25.444777 13093 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:25.445106 13093 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:25.449229 13093 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b: Sending DeleteTablet for 3 replicas of tablet 96e922f27c0c433b89f9dc3627f56670
I20260619 14:56:25.450600 13369 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e922f27c0c433b89f9dc3627f56670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:25 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:36392
I20260619 14:56:25.450594 13508 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e922f27c0c433b89f9dc3627f56670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:25 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:56578
I20260619 14:56:25.450627 13230 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e922f27c0c433b89f9dc3627f56670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:25 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:44206
I20260619 14:56:25.456066 13093 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:56:25.458201 13093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:25.467366 13622 tablet_replica.cc:330] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: stopping tablet replica
I20260619 14:56:25.467902 13622 raft_consensus.cc:2229] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:56:25.468608 13622 raft_consensus.cc:2258] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:25.469460 13623 tablet_replica.cc:330] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: stopping tablet replica
I20260619 14:56:25.469923 13623 raft_consensus.cc:2229] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:25.470427 13623 raft_consensus.cc:2258] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:25.470839 13368 tablet_service.cc:1467] Processing CreateTablet for tablet 04e6792632754d1b9c8d32fbd011c611 (DEFAULT_TABLE table=test-table [id=daf6503439c049e797bb753fdadcfd29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:25.471485 13368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04e6792632754d1b9c8d32fbd011c611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:25.472684 13623 ts_tablet_manager.cc:1905] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:25.473937 13621 tablet_replica.cc:330] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: stopping tablet replica
I20260619 14:56:25.474347 13621 raft_consensus.cc:2229] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:25.474888 13621 raft_consensus.cc:2258] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:25.477016 13621 ts_tablet_manager.cc:1905] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:25.479723 13230 tablet_service.cc:1467] Processing CreateTablet for tablet 04e6792632754d1b9c8d32fbd011c611 (DEFAULT_TABLE table=test-table [id=daf6503439c049e797bb753fdadcfd29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:25.479995 13623 ts_tablet_manager.cc:1918] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:56:25.480152 13623 log.cc:1196] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/96e922f27c0c433b89f9dc3627f56670
I20260619 14:56:25.480223 13230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04e6792632754d1b9c8d32fbd011c611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:25.481824 13622 ts_tablet_manager.cc:1905] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:25.482586 13597 tablet_bootstrap.cc:492] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251: Bootstrap starting.
I20260619 14:56:25.484362 13597 tablet_bootstrap.cc:654] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:25.486706 13623 ts_tablet_manager.cc:1939] T 96e922f27c0c433b89f9dc3627f56670 P 3d28479f2ed04696991b13fcd4640dc1: Deleting consensus metadata
I20260619 14:56:25.489054 13598 tablet_bootstrap.cc:492] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1: Bootstrap starting.
I20260619 14:56:25.489218 13079 catalog_manager.cc:4841] TS 3d28479f2ed04696991b13fcd4640dc1 (127.11.175.193:35217): tablet 96e922f27c0c433b89f9dc3627f56670 (table test-table [id=95ad3d640cc24c7c955ef9fe0ff70e42]) successfully deleted
I20260619 14:56:25.490592 13598 tablet_bootstrap.cc:654] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:25.490983 13597 tablet_bootstrap.cc:492] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251: No bootstrap required, opened a new log
I20260619 14:56:25.491148 13597 ts_tablet_manager.cc:1397] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260619 14:56:25.492064 13597 raft_consensus.cc:348] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.492305 13597 raft_consensus.cc:374] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:25.492403 13597 raft_consensus.cc:729] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55be6f806e1f48b8a69e2cf4bf6fe251, State: Initialized, Role: FOLLOWER
I20260619 14:56:25.492681 13598 tablet_bootstrap.cc:492] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1: No bootstrap required, opened a new log
I20260619 14:56:25.492810 13598 ts_tablet_manager.cc:1397] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260619 14:56:25.493613 13598 raft_consensus.cc:348] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.493824 13598 raft_consensus.cc:374] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:25.493916 13598 raft_consensus.cc:729] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d28479f2ed04696991b13fcd4640dc1, State: Initialized, Role: FOLLOWER
I20260619 14:56:25.493849 13597 consensus_queue.cc:260] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.494359 13598 consensus_queue.cc:260] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.495077 13621 ts_tablet_manager.cc:1918] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:56:25.495260 13621 log.cc:1196] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/96e922f27c0c433b89f9dc3627f56670
I20260619 14:56:25.495764 13598 ts_tablet_manager.cc:1428] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 14:56:25.495826 13621 ts_tablet_manager.cc:1939] T 96e922f27c0c433b89f9dc3627f56670 P 24fd0efba8a04c198d1ea5aeda73e95d: Deleting consensus metadata
I20260619 14:56:25.497670 13079 catalog_manager.cc:4841] TS 24fd0efba8a04c198d1ea5aeda73e95d (127.11.175.195:38449): tablet 96e922f27c0c433b89f9dc3627f56670 (table test-table [id=95ad3d640cc24c7c955ef9fe0ff70e42]) successfully deleted
I20260619 14:56:25.497799 13622 ts_tablet_manager.cc:1918] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:56:25.497977 13622 log.cc:1196] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/96e922f27c0c433b89f9dc3627f56670
I20260619 14:56:25.498556 13622 ts_tablet_manager.cc:1939] T 96e922f27c0c433b89f9dc3627f56670 P 55be6f806e1f48b8a69e2cf4bf6fe251: Deleting consensus metadata
I20260619 14:56:25.499645 13597 ts_tablet_manager.cc:1428] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251: Time spent starting tablet: real 0.008s user 0.001s sys 0.002s
I20260619 14:56:25.500358 13077 catalog_manager.cc:4841] TS 55be6f806e1f48b8a69e2cf4bf6fe251 (127.11.175.194:38631): tablet 96e922f27c0c433b89f9dc3627f56670 (table test-table [id=95ad3d640cc24c7c955ef9fe0ff70e42]) successfully deleted
I20260619 14:56:25.500480 13508 tablet_service.cc:1467] Processing CreateTablet for tablet 04e6792632754d1b9c8d32fbd011c611 (DEFAULT_TABLE table=test-table [id=daf6503439c049e797bb753fdadcfd29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:25.501631 13508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04e6792632754d1b9c8d32fbd011c611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:25.507093 13599 tablet_bootstrap.cc:492] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d: Bootstrap starting.
I20260619 14:56:25.509141 13599 tablet_bootstrap.cc:654] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:25.511219 13599 tablet_bootstrap.cc:492] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d: No bootstrap required, opened a new log
I20260619 14:56:25.511375 13599 ts_tablet_manager.cc:1397] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 14:56:25.511969 13599 raft_consensus.cc:348] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.512159 13599 raft_consensus.cc:374] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:25.512281 13599 raft_consensus.cc:729] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24fd0efba8a04c198d1ea5aeda73e95d, State: Initialized, Role: FOLLOWER
I20260619 14:56:25.512545 13599 consensus_queue.cc:260] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.513025 13599 ts_tablet_manager.cc:1428] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260619 14:56:25.802734 13605 raft_consensus.cc:482] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:25.803035 13605 raft_consensus.cc:504] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.805406 13605 leader_election.cc:288] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55be6f806e1f48b8a69e2cf4bf6fe251 (127.11.175.194:38631), 24fd0efba8a04c198d1ea5aeda73e95d (127.11.175.195:38449)
I20260619 14:56:25.815668 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.805922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:36589 (local address 127.11.175.195:38449)
0619 14:56:25.806115 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.806121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.807752 (+ 1631us) server_negotiation.cc:407] Connection header received
0619 14:56:25.808099 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.808106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.808196 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.808309 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:25.809080 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.810842 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.813669 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.814586 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.814815 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.814894 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.815264 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.815409 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 14:56:25.815788 13627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.805675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38449 (local address 127.11.175.193:36589)
0619 14:56:25.807553 (+ 1878us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:25.807579 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:25.807812 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:25.808332 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:25.808337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:25.808362 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:25.808850 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.808866 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.811041 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.811049 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.813386 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.813473 (+ 87us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.814840 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.814846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.815079 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.815121 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 14:56:25.815235 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 14:56:25.815394 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1687,"thread_start_us":1304,"threads_started":1}
I20260619 14:56:25.816903 13530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04e6792632754d1b9c8d32fbd011c611" candidate_uuid: "3d28479f2ed04696991b13fcd4640dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" is_pre_election: true
I20260619 14:56:25.817206 13530 raft_consensus.cc:2454] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d28479f2ed04696991b13fcd4640dc1 in term 0.
I20260619 14:56:25.817834 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.806400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:42207 (local address 127.11.175.194:38631)
0619 14:56:25.806583 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.806588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.807906 (+ 1318us) server_negotiation.cc:407] Connection header received
0619 14:56:25.808226 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.808232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.808348 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.808471 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:25.809240 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.810394 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.813576 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.815723 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.815958 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.816058 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.817304 (+ 1246us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.817480 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 14:56:25.817924 13182 leader_election.cc:302] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24fd0efba8a04c198d1ea5aeda73e95d, 3d28479f2ed04696991b13fcd4640dc1; no voters:
I20260619 14:56:25.818269 13628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.805741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:38631 (local address 127.11.175.193:42207)
0619 14:56:25.807795 (+ 2054us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:25.807814 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:25.807963 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:25.808505 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:25.808509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:25.808528 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:25.809076 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.809087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.810526 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.810534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.813309 (+ 2775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.813327 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.816794 (+ 3467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.816804 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.817020 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.817058 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 14:56:25.817259 (+ 201us) client_negotiation.cc:240] Negotiation successful
0619 14:56:25.817432 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1904,"mutex_wait_us":1,"thread_start_us":1667,"threads_started":1}
I20260619 14:56:25.818452 13605 raft_consensus.cc:2781] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:25.818614 13605 raft_consensus.cc:482] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:25.818760 13605 raft_consensus.cc:3037] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:25.819522 13394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04e6792632754d1b9c8d32fbd011c611" candidate_uuid: "3d28479f2ed04696991b13fcd4640dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" is_pre_election: true
I20260619 14:56:25.820091 13394 raft_consensus.cc:2454] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d28479f2ed04696991b13fcd4640dc1 in term 0.
I20260619 14:56:25.822016 13605 raft_consensus.cc:504] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.822839 13605 leader_election.cc:288] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [CANDIDATE]: Term 1 election: Requested vote from peers 55be6f806e1f48b8a69e2cf4bf6fe251 (127.11.175.194:38631), 24fd0efba8a04c198d1ea5aeda73e95d (127.11.175.195:38449)
I20260619 14:56:25.823305 13394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04e6792632754d1b9c8d32fbd011c611" candidate_uuid: "3d28479f2ed04696991b13fcd4640dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251"
I20260619 14:56:25.823552 13394 raft_consensus.cc:3037] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:25.823527 13530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04e6792632754d1b9c8d32fbd011c611" candidate_uuid: "3d28479f2ed04696991b13fcd4640dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24fd0efba8a04c198d1ea5aeda73e95d"
I20260619 14:56:25.823804 13530 raft_consensus.cc:3037] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:25.826037 13394 raft_consensus.cc:2454] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d28479f2ed04696991b13fcd4640dc1 in term 1.
I20260619 14:56:25.826507 13180 leader_election.cc:302] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d28479f2ed04696991b13fcd4640dc1, 55be6f806e1f48b8a69e2cf4bf6fe251; no voters:
I20260619 14:56:25.826833 13605 raft_consensus.cc:2781] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:25.827283 13530 raft_consensus.cc:2454] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d28479f2ed04696991b13fcd4640dc1 in term 1.
I20260619 14:56:25.827389 13605 raft_consensus.cc:686] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 LEADER]: Becoming Leader. State: Replica: 3d28479f2ed04696991b13fcd4640dc1, State: Running, Role: LEADER
I20260619 14:56:25.828172 13605 consensus_queue.cc:237] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.831866 13093 catalog_manager.cc:5495] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d28479f2ed04696991b13fcd4640dc1 (127.11.175.193). New cstate: current_term: 1 leader_uuid: "3d28479f2ed04696991b13fcd4640dc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } health_report { overall_health: UNKNOWN } } }
Jun 19 14:56:25 dist-test-slave-76rk krb5kdc[13048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880983, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:25.918097 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.889308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:33759 (local address 127.0.0.1:53652)
0619 14:56:25.890060 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:25.890081 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:25.890251 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:25.890716 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:25.890721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:25.891955 (+ 1234us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:25.892587 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.892602 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.893907 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.893914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.896033 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:25.896051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.897050 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.897061 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:25.897340 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.908249 (+ 10909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:25.908286 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:25.911684 (+ 3398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:25.915213 (+ 3529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:25.915227 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:25.915233 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:25.915734 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:25.916207 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:25.916211 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:25.916214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:25.916361 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:25.917185 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:25.917192 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:25.917383 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 14:56:25.917528 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 14:56:25.917779 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":386,"threads_started":1}
I20260619 14:56:25.918249 13637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:25.889409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.11.175.254:33759)
0619 14:56:25.890399 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:25.890403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:25.890424 (+ 21us) server_negotiation.cc:407] Connection header received
0619 14:56:25.890469 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:25.890473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:25.890534 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:25.890626 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:25.892830 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.893736 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.896346 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:25.896872 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:25.896982 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:25.911948 (+ 14966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:25.911954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:25.911957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:25.911996 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:25.915043 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:25.915902 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:25.915906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:25.915908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:25.916050 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:25.916581 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:25.916586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:25.916589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:25.917025 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:25.917148 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:25.917671 (+ 523us) server_negotiation.cc:299] Negotiation successful
0619 14:56:25.917907 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":896,"thread_start_us":755,"threads_started":1}
I20260619 14:56:25.924074 13093 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:53652:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 14:56:25.925062 13093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:25.932190 13230 tablet_service.cc:1467] Processing CreateTablet for tablet 18f77e78763a43d48acd9f02cdb20320 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b01643ab5ba643a5b3e7491f1a623e66]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 14:56:25.932387 13368 tablet_service.cc:1467] Processing CreateTablet for tablet 18f77e78763a43d48acd9f02cdb20320 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b01643ab5ba643a5b3e7491f1a623e66]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 14:56:25.932834 13230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f77e78763a43d48acd9f02cdb20320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:25.932890 13368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f77e78763a43d48acd9f02cdb20320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:25.938840 13508 tablet_service.cc:1467] Processing CreateTablet for tablet 18f77e78763a43d48acd9f02cdb20320 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b01643ab5ba643a5b3e7491f1a623e66]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 14:56:25.939438 13508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f77e78763a43d48acd9f02cdb20320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:25.940125 13597 tablet_bootstrap.cc:492] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: Bootstrap starting.
I20260619 14:56:25.940410 13598 tablet_bootstrap.cc:492] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: Bootstrap starting.
I20260619 14:56:25.941771 13597 tablet_bootstrap.cc:654] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:25.942229 13598 tablet_bootstrap.cc:654] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:25.943964 13597 tablet_bootstrap.cc:492] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: No bootstrap required, opened a new log
I20260619 14:56:25.944137 13597 ts_tablet_manager.cc:1397] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260619 14:56:25.945078 13598 tablet_bootstrap.cc:492] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: No bootstrap required, opened a new log
I20260619 14:56:25.945039 13597 raft_consensus.cc:348] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.945312 13598 ts_tablet_manager.cc:1397] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 14:56:25.945319 13597 raft_consensus.cc:374] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:25.945405 13597 raft_consensus.cc:729] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55be6f806e1f48b8a69e2cf4bf6fe251, State: Initialized, Role: FOLLOWER
I20260619 14:56:25.945664 13597 consensus_queue.cc:260] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.946206 13597 ts_tablet_manager.cc:1428] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 14:56:25.946090 13598 raft_consensus.cc:348] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.946092 13608 tablet_replica.cc:439] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } }
I20260619 14:56:25.946338 13598 raft_consensus.cc:374] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:25.946435 13608 tablet_replica.cc:442] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:25.946421 13598 raft_consensus.cc:729] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d28479f2ed04696991b13fcd4640dc1, State: Initialized, Role: FOLLOWER
I20260619 14:56:25.946712 13598 consensus_queue.cc:260] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.947144 13605 tablet_replica.cc:439] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } }
I20260619 14:56:25.947419 13605 tablet_replica.cc:442] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:25.947969 13598 ts_tablet_manager.cc:1428] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 14:56:25.948696 13599 tablet_bootstrap.cc:492] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: Bootstrap starting.
I20260619 14:56:25.950242 13599 tablet_bootstrap.cc:654] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:25.953584 13599 tablet_bootstrap.cc:492] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: No bootstrap required, opened a new log
I20260619 14:56:25.953881 13599 ts_tablet_manager.cc:1397] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260619 14:56:25.954888 13599 raft_consensus.cc:348] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.955247 13599 raft_consensus.cc:374] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:25.955420 13599 raft_consensus.cc:729] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24fd0efba8a04c198d1ea5aeda73e95d, State: Initialized, Role: FOLLOWER
I20260619 14:56:25.955739 13599 consensus_queue.cc:260] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:25.956238 13604 tablet_replica.cc:439] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } }
I20260619 14:56:25.956465 13599 ts_tablet_manager.cc:1428] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 14:56:25.956604 13604 tablet_replica.cc:442] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:26.047159 13608 raft_consensus.cc:482] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:26.047408 13608 raft_consensus.cc:504] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:26.048013 13608 leader_election.cc:288] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d28479f2ed04696991b13fcd4640dc1 (127.11.175.193:35217), 24fd0efba8a04c198d1ea5aeda73e95d (127.11.175.195:38449)
I20260619 14:56:26.048681 13530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f77e78763a43d48acd9f02cdb20320" candidate_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" is_pre_election: true
I20260619 14:56:26.048707 13255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f77e78763a43d48acd9f02cdb20320" candidate_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d28479f2ed04696991b13fcd4640dc1" is_pre_election: true
I20260619 14:56:26.048979 13530 raft_consensus.cc:2454] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55be6f806e1f48b8a69e2cf4bf6fe251 in term 0.
I20260619 14:56:26.048990 13255 raft_consensus.cc:2454] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55be6f806e1f48b8a69e2cf4bf6fe251 in term 0.
I20260619 14:56:26.049482 13322 leader_election.cc:302] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d28479f2ed04696991b13fcd4640dc1, 55be6f806e1f48b8a69e2cf4bf6fe251; no voters:
I20260619 14:56:26.049762 13608 raft_consensus.cc:2781] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:26.049876 13608 raft_consensus.cc:482] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:26.049995 13608 raft_consensus.cc:3037] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:26.052090 13608 raft_consensus.cc:504] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:26.052662 13608 leader_election.cc:288] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [CANDIDATE]: Term 1 election: Requested vote from peers 3d28479f2ed04696991b13fcd4640dc1 (127.11.175.193:35217), 24fd0efba8a04c198d1ea5aeda73e95d (127.11.175.195:38449)
I20260619 14:56:26.053254 13255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f77e78763a43d48acd9f02cdb20320" candidate_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d28479f2ed04696991b13fcd4640dc1"
I20260619 14:56:26.053272 13530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f77e78763a43d48acd9f02cdb20320" candidate_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24fd0efba8a04c198d1ea5aeda73e95d"
I20260619 14:56:26.053458 13255 raft_consensus.cc:3037] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:26.053551 13530 raft_consensus.cc:3037] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:26.055593 13255 raft_consensus.cc:2454] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55be6f806e1f48b8a69e2cf4bf6fe251 in term 1.
I20260619 14:56:26.055747 13530 raft_consensus.cc:2454] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55be6f806e1f48b8a69e2cf4bf6fe251 in term 1.
I20260619 14:56:26.056070 13322 leader_election.cc:302] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d28479f2ed04696991b13fcd4640dc1, 55be6f806e1f48b8a69e2cf4bf6fe251; no voters:
I20260619 14:56:26.056350 13608 raft_consensus.cc:2781] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:26.056725 13608 raft_consensus.cc:686] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 LEADER]: Becoming Leader. State: Replica: 55be6f806e1f48b8a69e2cf4bf6fe251, State: Running, Role: LEADER
I20260619 14:56:26.057076 13608 consensus_queue.cc:237] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } }
I20260619 14:56:26.058882 13641 tablet_replica.cc:439] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: TxnStatusTablet state changed. Reason: New leader 55be6f806e1f48b8a69e2cf4bf6fe251. Latest consensus state: current_term: 1 leader_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } }
I20260619 14:56:26.059203 13641 tablet_replica.cc:442] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: This TxnStatusTablet replica's current role is: LEADER
I20260619 14:56:26.060148 13643 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 14:56:26.060393 13643 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 14:56:26.060628 13092 catalog_manager.cc:5495] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55be6f806e1f48b8a69e2cf4bf6fe251 (127.11.175.194). New cstate: current_term: 1 leader_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260619 14:56:26.101876 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.096722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.11.175.194:38631)
0619 14:56:26.096933 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:26.096940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:26.097051 (+ 111us) server_negotiation.cc:407] Connection header received
0619 14:56:26.097350 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:26.097354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:26.097425 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:26.097533 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:26.097940 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.098789 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.100076 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.100513 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.100668 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.101140 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:26.101250 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:26.101350 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:26.101497 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 14:56:26.101578 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 14:56:26.101946 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.096586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:38631 (local address 127.0.0.1:36412)
0619 14:56:26.096945 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:26.096966 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:26.097147 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:26.097563 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:26.097568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:26.097581 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:56:26.097766 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:26.097775 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.098913 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.098922 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:26.099921 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:26.099933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.100702 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.100708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:26.100886 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.100947 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:56:26.101351 (+ 404us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:56:26.101377 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 14:56:26.101475 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 14:56:26.101626 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260619 14:56:26.106181 13255 raft_consensus.cc:1261] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 FOLLOWER]: Refusing update from remote peer 55be6f806e1f48b8a69e2cf4bf6fe251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:26.106240 13530 raft_consensus.cc:1261] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d [term 1 FOLLOWER]: Refusing update from remote peer 55be6f806e1f48b8a69e2cf4bf6fe251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:26.106884 13608 consensus_queue.cc:1035] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:26.107194 13641 consensus_queue.cc:1035] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:26.123574 13605 tablet_replica.cc:439] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: TxnStatusTablet state changed. Reason: New leader 55be6f806e1f48b8a69e2cf4bf6fe251. Latest consensus state: current_term: 1 leader_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } }
I20260619 14:56:26.123898 13605 tablet_replica.cc:442] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:26.137907 13605 tablet_replica.cc:439] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } }
I20260619 14:56:26.138258 13605 tablet_replica.cc:442] T 18f77e78763a43d48acd9f02cdb20320 P 3d28479f2ed04696991b13fcd4640dc1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:26.140583 13604 tablet_replica.cc:439] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: TxnStatusTablet state changed. Reason: New leader 55be6f806e1f48b8a69e2cf4bf6fe251. Latest consensus state: current_term: 1 leader_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } }
I20260619 14:56:26.140897 13604 tablet_replica.cc:442] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:26.151768 13608 tablet_replica.cc:439] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } }
I20260619 14:56:26.152237 13608 tablet_replica.cc:442] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: This TxnStatusTablet replica's current role is: LEADER
I20260619 14:56:26.155853 13608 tablet_replica.cc:439] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } }
I20260619 14:56:26.156204 13608 tablet_replica.cc:442] T 18f77e78763a43d48acd9f02cdb20320 P 55be6f806e1f48b8a69e2cf4bf6fe251: This TxnStatusTablet replica's current role is: LEADER
I20260619 14:56:26.160722 13604 tablet_replica.cc:439] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d28479f2ed04696991b13fcd4640dc1" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 35217 } } peers { permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 } } peers { permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 } } }
I20260619 14:56:26.161005 13604 tablet_replica.cc:442] T 18f77e78763a43d48acd9f02cdb20320 P 24fd0efba8a04c198d1ea5aeda73e95d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:26.171838 13655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.151698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44220 (local address 127.11.175.193:35217)
0619 14:56:26.166389 (+ 14691us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:26.166394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:26.166410 (+ 16us) server_negotiation.cc:407] Connection header received
0619 14:56:26.166494 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:26.166498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:26.166583 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:26.166802 (+ 219us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:26.167377 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.168142 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.169507 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.169927 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.170059 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.170514 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:26.171121 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:26.171299 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:26.171355 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 14:56:26.171438 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14572,"thread_start_us":14987,"threads_started":1}
I20260619 14:56:26.188045 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.182517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:35677 (local address 127.11.175.194:38631)
0619 14:56:26.182766 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:26.182774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:26.182811 (+ 37us) server_negotiation.cc:407] Connection header received
0619 14:56:26.183135 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:26.183140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:26.183220 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:26.183272 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:26.183846 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.184664 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.186140 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.186582 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.186816 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.187226 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:26.187376 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:26.187474 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:26.187618 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 14:56:26.187698 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 14:56:26.188125 13659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.182088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:38631 (local address 127.11.175.193:35677)
0619 14:56:26.182735 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:26.182759 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:26.182901 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:26.183430 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:26.183433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:26.183443 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:56:26.183696 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:26.183703 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.184819 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.184828 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:26.185989 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:26.186001 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.186845 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.186853 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:26.186986 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.187039 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:56:26.187491 (+ 452us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:56:26.187505 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 14:56:26.187596 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 14:56:26.187736 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":802,"threads_started":1}
I20260619 14:56:26.199165 13530 raft_consensus.cc:1261] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d [term 1 FOLLOWER]: Refusing update from remote peer 3d28479f2ed04696991b13fcd4640dc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:26.199476 13394 raft_consensus.cc:1261] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 FOLLOWER]: Refusing update from remote peer 3d28479f2ed04696991b13fcd4640dc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:26.200104 13629 consensus_queue.cc:1035] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24fd0efba8a04c198d1ea5aeda73e95d" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:26.200888 13605 consensus_queue.cc:1035] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55be6f806e1f48b8a69e2cf4bf6fe251" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 38631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:26.214876 13647 mvcc.cc:205] Tried to move back new op lower bound from 7298584519464669184 to 7298584517955743744. Current Snapshot: MvccSnapshot[applied={T|T < 7298584519464669184}]
I20260619 14:56:26.254094 13655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.241625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:38809 (local address 127.11.175.193:35217)
0619 14:56:26.241880 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:26.241891 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:26.247379 (+ 5488us) server_negotiation.cc:407] Connection header received
0619 14:56:26.247598 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:26.247604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:26.247687 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:26.247794 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:26.248575 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.249906 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.252122 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.252810 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.252954 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.253374 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:26.253513 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:26.253607 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:26.253708 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 14:56:26.253797 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 14:56:26.254153 13665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.241504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:35217 (local address 127.11.175.194:38809)
0619 14:56:26.247249 (+ 5745us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:26.247269 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:26.247444 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:26.248014 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:26.248018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:26.248031 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:56:26.248384 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:26.248393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.250133 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.250138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:26.251911 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:26.251925 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.252968 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.252973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:26.253125 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.253185 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:56:26.253612 (+ 427us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:56:26.253621 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 14:56:26.253680 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 14:56:26.253791 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5510,"thread_start_us":5319,"threads_started":1}
I20260619 14:56:26.301797 13092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:26.302052 13092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:26.305873 13092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f197370119c423b8f33b3390efd3e1b: Sending DeleteTablet for 3 replicas of tablet 04e6792632754d1b9c8d32fbd011c611
I20260619 14:56:26.307004 13368 tablet_service.cc:1514] Processing DeleteTablet for tablet 04e6792632754d1b9c8d32fbd011c611 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:26 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:36392
I20260619 14:56:26.307031 13230 tablet_service.cc:1514] Processing DeleteTablet for tablet 04e6792632754d1b9c8d32fbd011c611 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:26 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:44206
I20260619 14:56:26.307729 13508 tablet_service.cc:1514] Processing DeleteTablet for tablet 04e6792632754d1b9c8d32fbd011c611 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:26 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:56578
I20260619 14:56:26.330725 13669 tablet_replica.cc:330] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251: stopping tablet replica
I20260619 14:56:26.330969 13669 raft_consensus.cc:2229] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:26.331293 13669 raft_consensus.cc:2258] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:26.333384 13669 ts_tablet_manager.cc:1905] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:26.337039 13671 tablet_replica.cc:330] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d: stopping tablet replica
I20260619 14:56:26.337436 13671 raft_consensus.cc:2229] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:26.337823 13671 raft_consensus.cc:2258] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:26.338414 13670 tablet_replica.cc:330] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1: stopping tablet replica
I20260619 14:56:26.339766 13671 ts_tablet_manager.cc:1905] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:26.340097 13670 raft_consensus.cc:2229] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:56:26.340787 13670 raft_consensus.cc:2258] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:26.347976 13670 ts_tablet_manager.cc:1905] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:26.349627 13669 ts_tablet_manager.cc:1918] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:56:26.349782 13669 log.cc:1196] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/04e6792632754d1b9c8d32fbd011c611
Jun 19 14:56:26 dist-test-slave-76rk krb5kdc[13048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880985, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:56:26.350314 13669 ts_tablet_manager.cc:1939] T 04e6792632754d1b9c8d32fbd011c611 P 55be6f806e1f48b8a69e2cf4bf6fe251: Deleting consensus metadata
I20260619 14:56:26.351555 13077 catalog_manager.cc:4841] TS 55be6f806e1f48b8a69e2cf4bf6fe251 (127.11.175.194:38631): tablet 04e6792632754d1b9c8d32fbd011c611 (table test-table [id=daf6503439c049e797bb753fdadcfd29]) successfully deleted
I20260619 14:56:26.356611 13670 ts_tablet_manager.cc:1918] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:56:26.359766 13671 ts_tablet_manager.cc:1918] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:56:26.360072 13671 log.cc:1196] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/04e6792632754d1b9c8d32fbd011c611
I20260619 14:56:26.360675 13671 ts_tablet_manager.cc:1939] T 04e6792632754d1b9c8d32fbd011c611 P 24fd0efba8a04c198d1ea5aeda73e95d: Deleting consensus metadata
I20260619 14:56:26.362062 13079 catalog_manager.cc:4841] TS 24fd0efba8a04c198d1ea5aeda73e95d (127.11.175.195:38449): tablet 04e6792632754d1b9c8d32fbd011c611 (table test-table [id=daf6503439c049e797bb753fdadcfd29]) successfully deleted
I20260619 14:56:26.362469 13670 log.cc:1196] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/04e6792632754d1b9c8d32fbd011c611
I20260619 14:56:26.363190 13670 ts_tablet_manager.cc:1939] T 04e6792632754d1b9c8d32fbd011c611 P 3d28479f2ed04696991b13fcd4640dc1: Deleting consensus metadata
I20260619 14:56:26.364562 13655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.328603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44230 (local address 127.11.175.193:35217)
0619 14:56:26.328845 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:26.328855 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:26.332112 (+ 3257us) server_negotiation.cc:407] Connection header received
0619 14:56:26.338829 (+ 6717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:26.338835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:26.338944 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:26.339035 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:26.341667 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.342893 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.344918 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.345632 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.345737 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.356020 (+ 10283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:26.356038 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:26.356059 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:26.356108 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:26.360561 (+ 4453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:26.361620 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:26.361630 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:26.361634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:26.361835 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:26.362336 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:26.362341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:26.362346 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:26.362823 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:26.363211 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:26.363824 (+ 613us) server_negotiation.cc:299] Negotiation successful
0619 14:56:26.364085 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 14:56:26.364913 13079 catalog_manager.cc:4841] TS 3d28479f2ed04696991b13fcd4640dc1 (127.11.175.193:35217): tablet 04e6792632754d1b9c8d32fbd011c611 (table test-table [id=daf6503439c049e797bb753fdadcfd29]) successfully deleted
W20260619 14:56:26.366223 13295 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44230
I20260619 14:56:26.394378 13637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.369600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53656 (local address 127.11.175.254:33759)
0619 14:56:26.369804 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:26.369809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:26.375265 (+ 5456us) server_negotiation.cc:407] Connection header received
0619 14:56:26.375465 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:26.375471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:26.375575 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:26.375693 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:26.377820 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.378958 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.381091 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.381838 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.381964 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.385829 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:26.385838 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:26.385844 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:26.385897 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:26.390225 (+ 4328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:26.391334 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:26.391341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:26.391347 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:26.391626 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:26.392236 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:26.392242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:26.392245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:26.392789 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:26.393146 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:26.393654 (+ 508us) server_negotiation.cc:299] Negotiation successful
0619 14:56:26.393965 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260619 14:56:26.395058 13092 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53656
I20260619 14:56:26.397691 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13164
I20260619 14:56:26.418349 13655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.400185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44236 (local address 127.11.175.193:35217)
0619 14:56:26.400448 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:26.400455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:26.400478 (+ 23us) server_negotiation.cc:407] Connection header received
0619 14:56:26.400556 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:26.400561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:26.400665 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:26.400774 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:26.402527 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.403711 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.405824 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.406622 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.406775 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.410055 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:26.410062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:26.410068 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:26.410113 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:26.414574 (+ 4461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:26.415585 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:26.415593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:26.415597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:26.415779 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:26.416245 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:26.416250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:26.416255 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:26.416630 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:26.416907 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:26.417307 (+ 400us) server_negotiation.cc:299] Negotiation successful
0619 14:56:26.417927 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 14:56:26.419000 13295 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44236
W20260619 14:56:26.419721 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13164: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 14:56:26.419941 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 13164
I20260619 14:56:26.446590 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13304
Jun 19 14:56:26 dist-test-slave-76rk krb5kdc[13048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880985, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:56:26.465301 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.447402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36428 (local address 127.11.175.194:38631)
0619 14:56:26.447598 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:26.447603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:26.447702 (+ 99us) server_negotiation.cc:407] Connection header received
0619 14:56:26.447892 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:26.447896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:26.447970 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:26.448098 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:26.449703 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.450568 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.452551 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.453237 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.453385 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.458734 (+ 5349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:26.458755 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:26.458776 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:26.458846 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:26.462022 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:26.462963 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:26.462971 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:26.462977 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:26.463161 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:26.463570 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:26.463575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:26.463579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:26.464026 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:26.464341 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:26.464685 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 14:56:26.464935 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 14:56:26.465946 13434 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36428
W20260619 14:56:26.466622 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13304: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 14:56:26.466864 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 13304
I20260619 14:56:26.491533 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13443
Jun 19 14:56:26 dist-test-slave-76rk krb5kdc[13048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880985, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:56:26.506762 13677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:26.492367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56588 (local address 127.11.175.195:38449)
0619 14:56:26.492833 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:26.492839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:26.492860 (+ 21us) server_negotiation.cc:407] Connection header received
0619 14:56:26.492921 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:26.492926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:26.493014 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:26.493125 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:26.494385 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.495203 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.496431 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:26.496901 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:26.497008 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:26.501300 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:26.501318 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:26.501333 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:26.501373 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:26.504255 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:26.504934 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:26.504941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:26.504944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:26.505069 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:26.505398 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:26.505401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:26.505403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:26.505736 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:26.505967 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:26.506259 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 14:56:26.506431 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":233,"threads_started":1}
W20260619 14:56:26.507488 13573 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56588
W20260619 14:56:26.508167 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13443: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 14:56:26.508297 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 13443
I20260619 14:56:26.527158 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13061
I20260619 14:56:26.528364 13133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:26.646132 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 13061
2026-06-19T14:56:26Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5008 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:56:26 dist-test-slave-76rk krb5kdc[13681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:56:26 dist-test-slave-76rk krb5kdc[13681](info): set up 2 sockets
Jun 19 14:56:26 dist-test-slave-76rk krb5kdc[13681](info): commencing operation
krb5kdc: starting...
W20260619 14:56:27.759085 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:56:27 dist-test-slave-76rk krb5kdc[13681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:56:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:56:27.963891 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:45347
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:45347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:28.296911 13694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:28.297286 13694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:28.297463 13694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:28.304945 13694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:28.305106 13694 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 14:56:28.305207 13694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:28.305326 13694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:28.305430 13694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:28.305522 13694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:56:28.316160 13694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:45347
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:45347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:28.317543 13694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:28.319031 13694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:28.331329 13700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:28.331383 13702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:28.331413 13699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:28.332440 13694 server_base.cc:1034] running on GCE node
I20260619 14:56:28.333523 13694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:28.335398 13694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:28.336730 13694 hybrid_clock.cc:648] HybridClock initialized: now 1781880988336649 us; error 48 us; skew 500 ppm
I20260619 14:56:28.337149 13694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:28 dist-test-slave-76rk krb5kdc[13681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880988, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:28.340584 13694 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:56:28.342056 13694 webserver.cc:460] Webserver started at http://127.11.175.254:38881/ using document root <none> and password file <none>
I20260619 14:56:28.342794 13694 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:28.342886 13694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:28.343150 13694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:28.346120 13694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "1657435936bf4af38823b5165ace53bb"
format_stamp: "Formatted at 2026-06-19 14:56:28 on dist-test-slave-76rk"
server_key: "30c83d511b0a73e9228f0a363d91865d"
server_key_iv: "6fcb921fc0b99f22015867225de7e742"
server_key_version: "encryptionkey@0"
I20260619 14:56:28.346954 13694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "1657435936bf4af38823b5165ace53bb"
format_stamp: "Formatted at 2026-06-19 14:56:28 on dist-test-slave-76rk"
server_key: "30c83d511b0a73e9228f0a363d91865d"
server_key_iv: "6fcb921fc0b99f22015867225de7e742"
server_key_version: "encryptionkey@0"
I20260619 14:56:28.352960 13694 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 14:56:28.356575 13709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:28.357942 13694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 14:56:28.358093 13694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "1657435936bf4af38823b5165ace53bb"
format_stamp: "Formatted at 2026-06-19 14:56:28 on dist-test-slave-76rk"
server_key: "30c83d511b0a73e9228f0a363d91865d"
server_key_iv: "6fcb921fc0b99f22015867225de7e742"
server_key_version: "encryptionkey@0"
I20260619 14:56:28.358276 13694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:28.384119 13694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:28.388707 13694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:28.388954 13694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:28.411998 13694 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:45347
I20260619 14:56:28.412031 13770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:45347 every 8 connection(s)
I20260619 14:56:28.413375 13694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:28.414072 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 13694
I20260619 14:56:28.414305 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:28.415132 11967 external_mini_cluster.cc:1428] Setting key 1ae2177b312059c308a5201c17bbac77
I20260619 14:56:28.417877 13771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 14:56:28 dist-test-slave-76rk krb5kdc[13681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:28.431491 13771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb: Bootstrap starting.
I20260619 14:56:28.435871 13771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:28.437114 13771 log.cc:824] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:28.440485 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:28.417272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56406 (local address 127.11.175.254:45347)
0619 14:56:28.418101 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:28.418113 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:28.418155 (+ 42us) server_negotiation.cc:407] Connection header received
0619 14:56:28.419837 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:28.419858 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:28.420240 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:28.420591 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:28.422459 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:28.423993 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:28.425802 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:28.426516 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:28.426753 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:28.431540 (+ 4787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:28.431552 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:28.431573 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:28.431623 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:28.435582 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:28.436407 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:28.436415 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:28.436424 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:28.436635 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:28.437018 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:28.437023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:28.437026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:28.437846 (+ 820us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:28.438075 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:28.438629 (+ 554us) server_negotiation.cc:299] Negotiation successful
0619 14:56:28.439142 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":366,"threads_started":1}
I20260619 14:56:28.440773 13771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb: No bootstrap required, opened a new log
I20260619 14:56:28.449054 13771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1657435936bf4af38823b5165ace53bb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45347 } }
I20260619 14:56:28.449435 13771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:28.449510 13771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1657435936bf4af38823b5165ace53bb, State: Initialized, Role: FOLLOWER
I20260619 14:56:28.450011 13771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1657435936bf4af38823b5165ace53bb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45347 } }
I20260619 14:56:28.450208 13771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:28.450307 13771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:28.450438 13771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:28.452770 13771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1657435936bf4af38823b5165ace53bb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45347 } }
I20260619 14:56:28.453305 13771 leader_election.cc:302] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1657435936bf4af38823b5165ace53bb; no voters:
I20260619 14:56:28.453892 13771 leader_election.cc:288] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:28.454051 13776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:28.454875 13776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [term 1 LEADER]: Becoming Leader. State: Replica: 1657435936bf4af38823b5165ace53bb, State: Running, Role: LEADER
I20260619 14:56:28.455439 13776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1657435936bf4af38823b5165ace53bb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45347 } }
I20260619 14:56:28.456105 13771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:28.458228 13778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1657435936bf4af38823b5165ace53bb. Latest consensus state: current_term: 1 leader_uuid: "1657435936bf4af38823b5165ace53bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1657435936bf4af38823b5165ace53bb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45347 } } }
I20260619 14:56:28.458691 13778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:28.459005 13777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1657435936bf4af38823b5165ace53bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1657435936bf4af38823b5165ace53bb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45347 } } }
I20260619 14:56:28.459421 13777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:28.462774 13782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:28.468602 13782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:28.478102 13782 catalog_manager.cc:1347] Generated new cluster ID: c87b72c4b6444624a477cd3541ceaaf2
I20260619 14:56:28.478245 13782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:28.493630 13782 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:28.495719 13782 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:28.518291 13782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb: Generated new TSK 0
I20260619 14:56:28.519496 13782 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:56:28.582211 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45347
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:28.922299 13797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:28.922640 13797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:28.922850 13797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:28.930349 13797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:28.930511 13797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:28.930739 13797 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 14:56:28.930836 13797 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 14:56:28.930950 13797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:56:28.941059 13797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.175.254:45347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:28.942421 13797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:28.943787 13797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:28.957180 13803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:28.957192 13805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:28.957736 13802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:28.958487 13797 server_base.cc:1034] running on GCE node
I20260619 14:56:28.959167 13797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:28.960105 13797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:28.961308 13797 hybrid_clock.cc:648] HybridClock initialized: now 1781880988961265 us; error 41 us; skew 500 ppm
I20260619 14:56:28.961640 13797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:28 dist-test-slave-76rk krb5kdc[13681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880988, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:28.965040 13797 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:56:28.966697 13797 webserver.cc:460] Webserver started at http://127.11.175.193:34929/ using document root <none> and password file <none>
I20260619 14:56:28.967377 13797 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:28.967522 13797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:28.967846 13797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:28.970999 13797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "9b75b22f83244a5093d07fc0d2037b3e"
format_stamp: "Formatted at 2026-06-19 14:56:28 on dist-test-slave-76rk"
server_key: "25ea1c4aa64eedd5047f3648ffd8e302"
server_key_iv: "7270dff6c1500d1b747204d84dcafd0e"
server_key_version: "encryptionkey@0"
I20260619 14:56:28.971769 13797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "9b75b22f83244a5093d07fc0d2037b3e"
format_stamp: "Formatted at 2026-06-19 14:56:28 on dist-test-slave-76rk"
server_key: "25ea1c4aa64eedd5047f3648ffd8e302"
server_key_iv: "7270dff6c1500d1b747204d84dcafd0e"
server_key_version: "encryptionkey@0"
I20260619 14:56:28.977083 13797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:56:28.980809 13812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:28.982214 13797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 14:56:28.982425 13797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "9b75b22f83244a5093d07fc0d2037b3e"
format_stamp: "Formatted at 2026-06-19 14:56:28 on dist-test-slave-76rk"
server_key: "25ea1c4aa64eedd5047f3648ffd8e302"
server_key_iv: "7270dff6c1500d1b747204d84dcafd0e"
server_key_version: "encryptionkey@0"
I20260619 14:56:28.982718 13797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:29.004437 13797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:29.009133 13797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:29.009454 13797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:29.012355 13797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:29.012519 13797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:29.012673 13797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:29.012856 13797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Jun 19 14:56:29 dist-test-slave-76rk krb5kdc[13681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880988, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:29.039798 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:29.012916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:35997 (local address 127.11.175.254:45347)
0619 14:56:29.013781 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:29.013788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:29.014889 (+ 1101us) server_negotiation.cc:407] Connection header received
0619 14:56:29.018133 (+ 3244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:29.018141 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:29.018233 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:29.018399 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:29.020950 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.021960 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.025470 (+ 3510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.025913 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.026023 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:29.032045 (+ 6022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:29.032050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:29.032054 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:29.032090 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:29.034873 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:29.035960 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:29.035965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:29.035967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:29.036144 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:29.036819 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:29.036829 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:29.036834 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:29.037329 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:29.037510 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:29.038947 (+ 1437us) server_negotiation.cc:299] Negotiation successful
0619 14:56:29.039245 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":591,"threads_started":1}
I20260619 14:56:29.041858 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:29.013697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45347 (local address 127.11.175.193:35997)
0619 14:56:29.014678 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:29.014735 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:29.017717 (+ 2982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:29.018749 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:29.018758 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:29.019625 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:29.020703 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:29.020741 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.022191 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.022203 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:29.025258 (+ 3055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:29.025279 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.026218 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.026228 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:29.026607 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:29.027572 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:29.027619 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:29.031733 (+ 4114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:29.035050 (+ 3317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:29.035060 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:29.035079 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:29.035756 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:29.036270 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:29.036275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:29.036279 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:29.036615 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:29.037621 (+ 1006us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:29.037633 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:29.038358 (+ 725us) client_negotiation.cc:769] Sending connection context
0619 14:56:29.038781 (+ 423us) client_negotiation.cc:240] Negotiation successful
0619 14:56:29.039600 (+ 819us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":359,"threads_started":1}
I20260619 14:56:29.067430 13797 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:38601
I20260619 14:56:29.067437 13932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:38601 every 8 connection(s)
I20260619 14:56:29.069146 13797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:56:29.071605 13933 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45347
I20260619 14:56:29.071848 13933 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:29.072659 13933 heartbeater.cc:511] Master 127.11.175.254:45347 requested a full tablet report, sending...
I20260619 14:56:29.074144 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:29.056275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45347 (local address 127.11.175.193:47877)
0619 14:56:29.057095 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:29.057119 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:29.057364 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:29.057873 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:29.057882 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:29.058687 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:29.059290 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:29.059309 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.060449 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.060455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:29.063150 (+ 2695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:29.063177 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.064777 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.064784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:29.065023 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:29.065805 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:29.065858 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:29.067260 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:29.071098 (+ 3838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:29.071105 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:29.071109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:29.071620 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:29.072124 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:29.072128 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:29.072131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:29.072295 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:29.073021 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:29.073025 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:29.073226 (+ 201us) client_negotiation.cc:769] Sending connection context
0619 14:56:29.073388 (+ 162us) client_negotiation.cc:240] Negotiation successful
0619 14:56:29.073736 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":434,"threads_started":1}
I20260619 14:56:29.074165 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:29.056791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:47877 (local address 127.11.175.254:45347)
0619 14:56:29.057153 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:29.057159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:29.057263 (+ 104us) server_negotiation.cc:407] Connection header received
0619 14:56:29.057582 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:29.057588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:29.057661 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:29.057795 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:29.059468 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.060194 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.063408 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.064256 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.064375 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:29.067531 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:29.067537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:29.067542 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:29.067583 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:29.070955 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:29.071779 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:29.071784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:29.071787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:29.071971 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:29.072451 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:29.072455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:29.072458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:29.072897 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:29.073020 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:29.073552 (+ 532us) server_negotiation.cc:299] Negotiation successful
0619 14:56:29.073805 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260619 14:56:29.075253 13726 ts_manager.cc:194] Registered new tserver with Master: 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601)
I20260619 14:56:29.076787 13726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:35997
I20260619 14:56:29.078532 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 13797
I20260619 14:56:29.078760 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:56:29.079581 11967 external_mini_cluster.cc:1428] Setting key 0fc036608c64c7ff2e551c62d5f2c928
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:56:29.121986 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45347
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:29.456003 13937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:29.456429 13937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:29.456629 13937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:29.464393 13937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:29.464574 13937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:29.464699 13937 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 14:56:29.464757 13937 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 14:56:29.464821 13937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:56:29.474910 13937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.175.254:45347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:29.476186 13937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:29.477442 13937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:29.493875 13943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:29.493808 13942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:29.494069 13945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:29.495029 13937 server_base.cc:1034] running on GCE node
I20260619 14:56:29.495704 13937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:29.496815 13937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:29.498016 13937 hybrid_clock.cc:648] HybridClock initialized: now 1781880989497969 us; error 48 us; skew 500 ppm
I20260619 14:56:29.498342 13937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:29 dist-test-slave-76rk krb5kdc[13681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880989, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:29.501624 13937 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:56:29.503258 13937 webserver.cc:460] Webserver started at http://127.11.175.194:45711/ using document root <none> and password file <none>
I20260619 14:56:29.503945 13937 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:29.504072 13937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:29.504392 13937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:29.507387 13937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259"
format_stamp: "Formatted at 2026-06-19 14:56:29 on dist-test-slave-76rk"
server_key: "ad48e6661e3c9dcab93cca17de1b7a85"
server_key_iv: "d42c3226db90b25f474e6b9affc543a7"
server_key_version: "encryptionkey@0"
I20260619 14:56:29.508138 13937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259"
format_stamp: "Formatted at 2026-06-19 14:56:29 on dist-test-slave-76rk"
server_key: "ad48e6661e3c9dcab93cca17de1b7a85"
server_key_iv: "d42c3226db90b25f474e6b9affc543a7"
server_key_version: "encryptionkey@0"
I20260619 14:56:29.513531 13937 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 14:56:29.517295 13952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:29.518929 13937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:56:29.519112 13937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259"
format_stamp: "Formatted at 2026-06-19 14:56:29 on dist-test-slave-76rk"
server_key: "ad48e6661e3c9dcab93cca17de1b7a85"
server_key_iv: "d42c3226db90b25f474e6b9affc543a7"
server_key_version: "encryptionkey@0"
I20260619 14:56:29.519331 13937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:29.540191 13937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:29.544495 13937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:29.544795 13937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:29.547727 13937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:29.547876 13937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:29.548003 13937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:29.548095 13937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 14:56:29 dist-test-slave-76rk krb5kdc[13681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880989, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:29.577875 13961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:29.548963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45347 (local address 127.11.175.194:50763)
0619 14:56:29.549875 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:29.549922 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:29.552501 (+ 2579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:29.553524 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:29.553541 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:29.554602 (+ 1061us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:29.555704 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:29.555743 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.557254 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.557261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:29.559438 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:29.559458 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.561023 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.561036 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:29.561454 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:29.562516 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:29.562545 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:29.566376 (+ 3831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:29.571077 (+ 4701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:29.571088 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:29.571108 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:29.571951 (+ 843us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:29.572488 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:29.572491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:29.572493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:29.572832 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:29.573734 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:29.573743 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:29.574451 (+ 708us) client_negotiation.cc:769] Sending connection context
0619 14:56:29.574934 (+ 483us) client_negotiation.cc:240] Negotiation successful
0619 14:56:29.575743 (+ 809us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":388,"threads_started":1}
I20260619 14:56:29.579598 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:29.548338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:50763 (local address 127.11.175.254:45347)
0619 14:56:29.548549 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:29.548556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:29.550075 (+ 1519us) server_negotiation.cc:407] Connection header received
0619 14:56:29.552936 (+ 2861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:29.552942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:29.553064 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:29.553189 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:29.555957 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.557013 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.559617 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.560423 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.560577 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:29.566716 (+ 6139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:29.566722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:29.566726 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:29.566761 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:29.570837 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:29.572159 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:29.572167 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:29.572171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:29.572324 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:29.573018 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:29.573026 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:29.573031 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:29.573426 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:29.573635 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:29.578783 (+ 5148us) server_negotiation.cc:299] Negotiation successful
0619 14:56:29.579137 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 14:56:29.593647 13937 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:42037
I20260619 14:56:29.593694 14071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:42037 every 8 connection(s)
I20260619 14:56:29.595330 13937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:56:29.597817 14072 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45347
I20260619 14:56:29.598050 14072 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:29.598790 14072 heartbeater.cc:511] Master 127.11.175.254:45347 requested a full tablet report, sending...
I20260619 14:56:29.600271 13726 ts_manager.cc:194] Registered new tserver with Master: 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037)
I20260619 14:56:29.601368 13726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:50763
I20260619 14:56:29.602941 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 13937
I20260619 14:56:29.603120 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:56:29.603683 11967 external_mini_cluster.cc:1428] Setting key 8762cc4c3416b7e09316e03df43150af
I20260619 14:56:29.603737 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:29.586913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:41333 (local address 127.11.175.254:45347)
0619 14:56:29.587166 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:29.587173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:29.588180 (+ 1007us) server_negotiation.cc:407] Connection header received
0619 14:56:29.588412 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:29.588420 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:29.588522 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:29.588657 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:29.590558 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.591917 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.594303 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.594813 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.594940 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:29.597399 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:29.597404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:29.597408 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:29.597442 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:29.600648 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:29.601441 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:29.601446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:29.601449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:29.601692 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:29.602104 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:29.602111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:29.602114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:29.602445 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:29.602599 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:29.603119 (+ 520us) server_negotiation.cc:299] Negotiation successful
0619 14:56:29.603378 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 14:56:29.604105 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:29.587174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45347 (local address 127.11.175.194:41333)
0619 14:56:29.588028 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:29.588054 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:29.588252 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:29.588877 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:29.588883 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:29.589717 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:29.590320 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:29.590346 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.592080 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.592086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:29.594098 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:29.594116 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:29.595018 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:29.595025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:29.595208 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:29.595922 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:29.595959 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:29.597137 (+ 1178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:29.600779 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:29.600785 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:29.600789 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:29.601289 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:29.601831 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:29.601837 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:29.601841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:29.601987 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:29.602576 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:29.602581 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:29.602872 (+ 291us) client_negotiation.cc:769] Sending connection context
0619 14:56:29.603023 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 14:56:29.603456 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":2,"thread_start_us":506,"threads_started":1}
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:56:29.646680 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45347
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:29.983685 14076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:29.984041 14076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:29.984236 14076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:29.992461 14076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:29.992633 14076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:29.992785 14076 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 14:56:29.992877 14076 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 14:56:29.993011 14076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:56:30.003087 14076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.175.254:45347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:30.004462 14076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:30.006479 14076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:30.021648 14081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:30.021548 14084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:30.021590 14082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:30.023586 14076 server_base.cc:1034] running on GCE node
I20260619 14:56:30.024390 14076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:30.025514 14076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:30.026747 14076 hybrid_clock.cc:648] HybridClock initialized: now 1781880990026690 us; error 53 us; skew 500 ppm
I20260619 14:56:30.027067 14076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:30 dist-test-slave-76rk krb5kdc[13681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880990, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:30.031231 14076 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:56:30.032953 14076 webserver.cc:460] Webserver started at http://127.11.175.195:34377/ using document root <none> and password file <none>
I20260619 14:56:30.033689 14076 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:30.033808 14076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:30.034130 14076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:30.037369 14076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "c4390a811a4a40a4a1bd497ba48a7c6f"
format_stamp: "Formatted at 2026-06-19 14:56:30 on dist-test-slave-76rk"
server_key: "7bbadb67e3faa6bb1d96b2bbb62a014a"
server_key_iv: "769e1f0bb979d196ced966255d8fe033"
server_key_version: "encryptionkey@0"
I20260619 14:56:30.038208 14076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "c4390a811a4a40a4a1bd497ba48a7c6f"
format_stamp: "Formatted at 2026-06-19 14:56:30 on dist-test-slave-76rk"
server_key: "7bbadb67e3faa6bb1d96b2bbb62a014a"
server_key_iv: "769e1f0bb979d196ced966255d8fe033"
server_key_version: "encryptionkey@0"
I20260619 14:56:30.044021 14076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 14:56:30.048020 14091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:30.049499 14076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 14:56:30.049646 14076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "c4390a811a4a40a4a1bd497ba48a7c6f"
format_stamp: "Formatted at 2026-06-19 14:56:30 on dist-test-slave-76rk"
server_key: "7bbadb67e3faa6bb1d96b2bbb62a014a"
server_key_iv: "769e1f0bb979d196ced966255d8fe033"
server_key_version: "encryptionkey@0"
I20260619 14:56:30.049885 14076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:30.067356 14076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:30.072813 14076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:30.073149 14076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:30.076119 14076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:30.076261 14076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:30.076390 14076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:30.076488 14076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:30.080351 13933 heartbeater.cc:503] Master 127.11.175.254:45347 was elected leader, sending a full tablet report...
Jun 19 14:56:30 dist-test-slave-76rk krb5kdc[13681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880990, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:30.101794 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.076757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:38987 (local address 127.11.175.254:45347)
0619 14:56:30.076975 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.076982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.078559 (+ 1577us) server_negotiation.cc:407] Connection header received
0619 14:56:30.082173 (+ 3614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.082183 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.082310 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.082392 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:30.085825 (+ 3433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.087007 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.088841 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.089246 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.089385 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.094313 (+ 4928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:30.094319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:30.094322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:30.094360 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:30.097873 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:30.098727 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:30.098732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:30.098734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:30.098874 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:30.099394 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:30.099399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:30.099405 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:30.099747 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:30.099914 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.101263 (+ 1349us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.101521 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 14:56:30.104241 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.077434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45347 (local address 127.11.175.195:38987)
0619 14:56:30.078338 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.078407 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.081737 (+ 3330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.083127 (+ 1390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.083144 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.084241 (+ 1097us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:30.085518 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.085557 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.087201 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.087206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.088668 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.088681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.089505 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.089513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.089745 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.090505 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:30.090529 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:30.094009 (+ 3480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:30.098036 (+ 4027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:30.098047 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:30.098066 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:30.098548 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:30.099003 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:30.099009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:30.099012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:30.099253 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:30.099945 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:30.099959 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:30.100650 (+ 691us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.101073 (+ 423us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.101884 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":337,"threads_started":1}
I20260619 14:56:30.132653 14076 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:37309
I20260619 14:56:30.132654 14210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:37309 every 8 connection(s)
I20260619 14:56:30.134291 14076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:56:30.135927 14177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.118793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45347 (local address 127.11.175.195:35599)
0619 14:56:30.119762 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.119783 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.119927 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.120349 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.120356 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.121136 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:30.121700 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.121722 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.123408 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.123414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.124746 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.124771 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.126034 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.126045 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.126286 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.127172 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:30.127207 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:30.128616 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:30.132968 (+ 4352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:30.132972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:30.132975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:30.133400 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:30.133834 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:30.133837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:30.133840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:30.133962 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:30.134582 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:30.134585 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:30.134815 (+ 230us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.135040 (+ 225us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.135445 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"thread_start_us":678,"threads_started":1}
I20260619 14:56:30.136039 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.118792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:35599 (local address 127.11.175.254:45347)
0619 14:56:30.119007 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.119016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.119912 (+ 896us) server_negotiation.cc:407] Connection header received
0619 14:56:30.120066 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.120075 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.120176 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.120308 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:30.121933 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.123193 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.124935 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.125795 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.125971 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.128875 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:30.128881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:30.128887 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:30.128935 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:30.132848 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:30.133531 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:30.133537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:30.133540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:30.133719 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:30.134070 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:30.134075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:30.134078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:30.134456 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:30.134659 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.135164 (+ 505us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.135499 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 14:56:30.138476 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 14076
I20260619 14:56:30.138689 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:56:30.139266 11967 external_mini_cluster.cc:1428] Setting key 5190f14dc9d08c9137bc98919c002b60
I20260619 14:56:30.136817 14211 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45347
I20260619 14:56:30.140026 14211 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:30.140846 14211 heartbeater.cc:511] Master 127.11.175.254:45347 requested a full tablet report, sending...
I20260619 14:56:30.142248 13726 ts_manager.cc:194] Registered new tserver with Master: c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195:37309)
I20260619 14:56:30.143121 13726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:38987
I20260619 14:56:30.154950 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 14:56:30 dist-test-slave-76rk krb5kdc[13681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 14:56:30 dist-test-slave-76rk krb5kdc[13681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:30.205122 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.190625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56422 (local address 127.11.175.254:45347)
0619 14:56:30.190918 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.190926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.191173 (+ 247us) server_negotiation.cc:407] Connection header received
0619 14:56:30.191423 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.191428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.191513 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.191627 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:30.193085 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.193817 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.195217 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.195651 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.195802 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.199756 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:30.199761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:30.199765 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:30.199810 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:30.202357 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:30.203097 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:30.203109 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:30.203113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:30.203239 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:30.203640 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:30.203644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:30.203646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:30.203937 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:30.204204 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.204531 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.204783 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 14:56:30.209697 13726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:56:30.216650 13726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:30.256273 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.231575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.11.175.194:42037)
0619 14:56:30.240227 (+ 8652us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.240236 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.247219 (+ 6983us) server_negotiation.cc:407] Connection header received
0619 14:56:30.247360 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.247369 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.247640 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.247847 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:30.249148 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.251223 (+ 2075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.253444 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.254689 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.254868 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.255069 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.255797 (+ 728us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.255983 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8495,"thread_start_us":8363,"threads_started":1}
I20260619 14:56:30.256563 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.230036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53364 (local address 127.11.175.195:37309)
0619 14:56:30.239771 (+ 9735us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.239784 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.247108 (+ 7324us) server_negotiation.cc:407] Connection header received
0619 14:56:30.247379 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.247385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.247655 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.247843 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:30.249250 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.251001 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.253683 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.255226 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.255466 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.255632 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.255901 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.256059 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9483,"thread_start_us":9183,"threads_started":1}
I20260619 14:56:30.256587 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.229708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40260 (local address 127.11.175.193:38601)
0619 14:56:30.246321 (+ 16613us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.246331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.247333 (+ 1002us) server_negotiation.cc:407] Connection header received
0619 14:56:30.247427 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.247432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.247628 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.247847 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:30.249139 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.250370 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.253209 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.254843 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.255033 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.255217 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.256110 (+ 893us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.256278 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16362,"thread_start_us":16164,"threads_started":1}
I20260619 14:56:30.256999 14228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.229550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:37309 (local address 127.0.0.1:53364)
0619 14:56:30.246867 (+ 17317us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.246889 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.247110 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.248118 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.248122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.248142 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:30.248999 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.249017 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.251148 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.251153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.253427 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.253442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.255465 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.255472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.255651 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.255688 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.255846 (+ 158us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.255986 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17182,"mutex_wait_us":127,"spinlock_wait_cycles":22400,"thread_start_us":17079,"threads_started":1}
I20260619 14:56:30.257184 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.230903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:42037 (local address 127.0.0.1:46954)
0619 14:56:30.246918 (+ 16015us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.246936 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.247109 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.248093 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.248101 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.248131 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:30.248862 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.248878 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.251363 (+ 2485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.251371 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.253246 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.253259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.255198 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.255204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.255428 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.255510 (+ 82us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.255641 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.256626 (+ 985us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15875,"mutex_wait_us":855,"spinlock_wait_cycles":3200,"thread_start_us":15981,"threads_started":1}
I20260619 14:56:30.257869 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.229493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38601 (local address 127.0.0.1:40260)
0619 14:56:30.246763 (+ 17270us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.246835 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.247109 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.247857 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.247865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.247912 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:30.248854 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.248869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.250512 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.250518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.252999 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.253021 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.255098 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.255108 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.255414 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.255507 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.255641 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.255867 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16725,"mutex_wait_us":255,"spinlock_wait_cycles":14336,"thread_start_us":14530,"threads_started":1}
I20260619 14:56:30.260805 14141 tablet_service.cc:1467] Processing CreateTablet for tablet d9e794d46a31489ca211398cbe3a49b0 (DEFAULT_TABLE table=test-table [id=a144bf4f62eb471f9870e0ca46a5a799]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:30.260908 13863 tablet_service.cc:1467] Processing CreateTablet for tablet d9e794d46a31489ca211398cbe3a49b0 (DEFAULT_TABLE table=test-table [id=a144bf4f62eb471f9870e0ca46a5a799]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:30.262009 13863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e794d46a31489ca211398cbe3a49b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:30.262086 14141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e794d46a31489ca211398cbe3a49b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:30.264562 14002 tablet_service.cc:1467] Processing CreateTablet for tablet d9e794d46a31489ca211398cbe3a49b0 (DEFAULT_TABLE table=test-table [id=a144bf4f62eb471f9870e0ca46a5a799]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:30.265951 14002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e794d46a31489ca211398cbe3a49b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:30.273967 14230 tablet_bootstrap.cc:492] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: Bootstrap starting.
I20260619 14:56:30.277626 14230 tablet_bootstrap.cc:654] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:30.278749 14230 log.cc:824] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:30.279682 14231 tablet_bootstrap.cc:492] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: Bootstrap starting.
I20260619 14:56:30.282374 14233 tablet_bootstrap.cc:492] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Bootstrap starting.
I20260619 14:56:30.283423 14230 tablet_bootstrap.cc:492] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: No bootstrap required, opened a new log
I20260619 14:56:30.283720 14230 ts_tablet_manager.cc:1397] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260619 14:56:30.283794 14231 tablet_bootstrap.cc:654] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:30.285158 14231 log.cc:824] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:30.285264 14233 tablet_bootstrap.cc:654] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:30.286958 14233 log.cc:824] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:30.289952 14233 tablet_bootstrap.cc:492] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: No bootstrap required, opened a new log
I20260619 14:56:30.290256 14233 ts_tablet_manager.cc:1397] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260619 14:56:30.292575 14231 tablet_bootstrap.cc:492] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: No bootstrap required, opened a new log
I20260619 14:56:30.292881 14231 ts_tablet_manager.cc:1397] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.002s
I20260619 14:56:30.293742 14230 raft_consensus.cc:348] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } }
I20260619 14:56:30.294206 14230 raft_consensus.cc:374] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:30.294312 14230 raft_consensus.cc:729] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b75b22f83244a5093d07fc0d2037b3e, State: Initialized, Role: FOLLOWER
I20260619 14:56:30.294998 14230 consensus_queue.cc:260] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } }
I20260619 14:56:30.296757 14230 ts_tablet_manager.cc:1428] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260619 14:56:30.300639 14233 raft_consensus.cc:348] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } }
I20260619 14:56:30.301082 14233 raft_consensus.cc:374] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:30.301179 14233 raft_consensus.cc:729] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab8b1a7ae8440efaf25ec4a7fad2259, State: Initialized, Role: FOLLOWER
I20260619 14:56:30.301796 14233 consensus_queue.cc:260] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } }
I20260619 14:56:30.303015 14072 heartbeater.cc:503] Master 127.11.175.254:45347 was elected leader, sending a full tablet report...
I20260619 14:56:30.303735 14233 ts_tablet_manager.cc:1428] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260619 14:56:30.303855 14231 raft_consensus.cc:348] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } }
I20260619 14:56:30.304271 14231 raft_consensus.cc:374] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:30.304409 14231 raft_consensus.cc:729] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4390a811a4a40a4a1bd497ba48a7c6f, State: Initialized, Role: FOLLOWER
I20260619 14:56:30.304999 14231 consensus_queue.cc:260] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } }
I20260619 14:56:30.306192 14211 heartbeater.cc:503] Master 127.11.175.254:45347 was elected leader, sending a full tablet report...
I20260619 14:56:30.306783 14231 ts_tablet_manager.cc:1428] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
W20260619 14:56:30.321971 13934 tablet.cc:2369] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:56:30.344518 14238 raft_consensus.cc:482] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:30.344938 14238 raft_consensus.cc:504] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } }
I20260619 14:56:30.347246 14238 leader_election.cc:288] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601), 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037)
W20260619 14:56:30.348021 14073 tablet.cc:2369] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:56:30.355237 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.348567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:52495 (local address 127.11.175.193:38601)
0619 14:56:30.348775 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.348780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.348799 (+ 19us) server_negotiation.cc:407] Connection header received
0619 14:56:30.348919 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.348924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.348997 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.349092 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:30.349498 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.350687 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.353072 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.354288 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.354438 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.354511 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.354854 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.354997 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 14:56:30.355299 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.347685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38601 (local address 127.11.175.195:52495)
0619 14:56:30.348159 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.348185 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.348316 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.349114 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.349118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.349154 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:30.349344 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.349355 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.350818 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.350823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.352921 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.352942 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.354430 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.354438 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.354686 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.354734 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.354828 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.354978 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302}
I20260619 14:56:30.355896 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.347677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:35337 (local address 127.11.175.194:42037)
0619 14:56:30.348237 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.348244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.349225 (+ 981us) server_negotiation.cc:407] Connection header received
0619 14:56:30.349304 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.349310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.349380 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.349470 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:30.350085 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.351136 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.353450 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.354673 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.354799 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.354872 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.355485 (+ 613us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.355646 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412}
I20260619 14:56:30.356544 14239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.347841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:42037 (local address 127.11.175.195:35337)
0619 14:56:30.348555 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.348574 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.348707 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.349495 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.349500 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.349526 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:30.349905 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.349917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.351297 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.351302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.353291 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.353304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.355105 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.355111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.355277 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.355315 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.356108 (+ 793us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.356242 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":402,"threads_started":1}
I20260619 14:56:30.356766 13883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e794d46a31489ca211398cbe3a49b0" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b75b22f83244a5093d07fc0d2037b3e" is_pre_election: true
I20260619 14:56:30.357417 13883 raft_consensus.cc:2454] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 0.
I20260619 14:56:30.357610 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e794d46a31489ca211398cbe3a49b0" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" is_pre_election: true
I20260619 14:56:30.358125 14022 raft_consensus.cc:2454] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 0.
I20260619 14:56:30.358331 14095 leader_election.cc:302] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b75b22f83244a5093d07fc0d2037b3e, c4390a811a4a40a4a1bd497ba48a7c6f; no voters:
I20260619 14:56:30.359195 14238 raft_consensus.cc:2781] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:30.359371 14238 raft_consensus.cc:482] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:30.359491 14238 raft_consensus.cc:3037] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:30.363225 14238 raft_consensus.cc:504] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } }
I20260619 14:56:30.364504 14238 leader_election.cc:288] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 election: Requested vote from peers 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601), 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037)
I20260619 14:56:30.364549 13883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e794d46a31489ca211398cbe3a49b0" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b75b22f83244a5093d07fc0d2037b3e"
I20260619 14:56:30.364632 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e794d46a31489ca211398cbe3a49b0" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259"
I20260619 14:56:30.364753 13883 raft_consensus.cc:3037] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:30.364849 14022 raft_consensus.cc:3037] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:30.366988 13883 raft_consensus.cc:2454] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 1.
I20260619 14:56:30.367372 14022 raft_consensus.cc:2454] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 1.
I20260619 14:56:30.367439 14095 leader_election.cc:302] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b75b22f83244a5093d07fc0d2037b3e, c4390a811a4a40a4a1bd497ba48a7c6f; no voters:
I20260619 14:56:30.367762 14238 raft_consensus.cc:2781] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:30.368283 14238 raft_consensus.cc:686] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 LEADER]: Becoming Leader. State: Replica: c4390a811a4a40a4a1bd497ba48a7c6f, State: Running, Role: LEADER
I20260619 14:56:30.368913 14238 consensus_queue.cc:237] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } }
I20260619 14:56:30.374228 13726 catalog_manager.cc:5495] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f reported cstate change: term changed from 0 to 1, leader changed from <none> to c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195). New cstate: current_term: 1 leader_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } health_report { overall_health: UNKNOWN } } }
W20260619 14:56:30.387539 14212 tablet.cc:2369] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 14:56:30 dist-test-slave-76rk krb5kdc[13681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880988, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:30.447371 14247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.433295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45347 (local address 127.0.0.1:56426)
0619 14:56:30.433959 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.433974 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.434163 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.434512 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.434516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.435339 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:30.435644 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.435654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.436664 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.436671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.437624 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.437636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.438363 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.438370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.438557 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.439255 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:30.439271 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:30.441587 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:30.444778 (+ 3191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:30.444790 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:30.444796 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:30.445252 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:30.445677 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:30.445682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:30.445691 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:30.445856 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:30.446369 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:30.446375 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:30.446532 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.446777 (+ 245us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.446999 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":362,"threads_started":1}
I20260619 14:56:30.447389 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.433273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56426 (local address 127.11.175.254:45347)
0619 14:56:30.433491 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.433496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.434073 (+ 577us) server_negotiation.cc:407] Connection header received
0619 14:56:30.434295 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.434304 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.434381 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.434486 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:30.435788 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.436538 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.437779 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.438181 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.438327 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.441794 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:30.441799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:30.441803 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:30.441834 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:30.444600 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:30.445385 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:30.445394 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:30.445396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:30.445527 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:30.445957 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:30.445961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:30.445963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:30.446260 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:30.446371 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.446868 (+ 497us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.447120 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":88}
I20260619 14:56:30.452440 13726 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:56426:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 14:56:30.453483 13726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:30.460749 13863 tablet_service.cc:1467] Processing CreateTablet for tablet d5f0ae59b866409b8cdc0fa099bdb001 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c3707b7f0574ab3b24c6e43f09bddf7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 14:56:30.460939 14141 tablet_service.cc:1467] Processing CreateTablet for tablet d5f0ae59b866409b8cdc0fa099bdb001 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c3707b7f0574ab3b24c6e43f09bddf7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 14:56:30.460827 14002 tablet_service.cc:1467] Processing CreateTablet for tablet d5f0ae59b866409b8cdc0fa099bdb001 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c3707b7f0574ab3b24c6e43f09bddf7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 14:56:30.461723 14141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f0ae59b866409b8cdc0fa099bdb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:30.461750 13863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f0ae59b866409b8cdc0fa099bdb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:30.461723 14002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f0ae59b866409b8cdc0fa099bdb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:30.468145 14230 tablet_bootstrap.cc:492] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: Bootstrap starting.
I20260619 14:56:30.468945 14231 tablet_bootstrap.cc:492] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: Bootstrap starting.
I20260619 14:56:30.469484 14233 tablet_bootstrap.cc:492] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Bootstrap starting.
I20260619 14:56:30.469869 14230 tablet_bootstrap.cc:654] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:30.470589 14231 tablet_bootstrap.cc:654] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:30.470721 14233 tablet_bootstrap.cc:654] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:30.477066 14231 tablet_bootstrap.cc:492] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: No bootstrap required, opened a new log
I20260619 14:56:30.477247 14231 ts_tablet_manager.cc:1397] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260619 14:56:30.477569 14233 tablet_bootstrap.cc:492] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: No bootstrap required, opened a new log
I20260619 14:56:30.477718 14233 ts_tablet_manager.cc:1397] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260619 14:56:30.477994 14231 raft_consensus.cc:348] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.478207 14231 raft_consensus.cc:374] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:30.478327 14231 raft_consensus.cc:729] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4390a811a4a40a4a1bd497ba48a7c6f, State: Initialized, Role: FOLLOWER
I20260619 14:56:30.478416 14233 raft_consensus.cc:348] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.478715 14233 raft_consensus.cc:374] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:30.478578 14231 consensus_queue.cc:260] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.478796 14233 raft_consensus.cc:729] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab8b1a7ae8440efaf25ec4a7fad2259, State: Initialized, Role: FOLLOWER
I20260619 14:56:30.479000 14233 consensus_queue.cc:260] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.479112 14238 tablet_replica.cc:439] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } }
I20260619 14:56:30.479264 14231 ts_tablet_manager.cc:1428] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 14:56:30.479410 14230 tablet_bootstrap.cc:492] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: No bootstrap required, opened a new log
I20260619 14:56:30.479413 14238 tablet_replica.cc:442] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:30.479542 14230 ts_tablet_manager.cc:1397] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260619 14:56:30.479437 14237 tablet_replica.cc:439] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } }
I20260619 14:56:30.479759 14237 tablet_replica.cc:442] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:30.480211 14233 ts_tablet_manager.cc:1428] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 14:56:30.480225 14230 raft_consensus.cc:348] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.480492 14230 raft_consensus.cc:374] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:30.480584 14230 raft_consensus.cc:729] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b75b22f83244a5093d07fc0d2037b3e, State: Initialized, Role: FOLLOWER
I20260619 14:56:30.480795 14230 consensus_queue.cc:260] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.481390 14230 ts_tablet_manager.cc:1428] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 14:56:30.481442 14236 tablet_replica.cc:439] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } }
I20260619 14:56:30.481860 14236 tablet_replica.cc:442] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:30.601886 14236 raft_consensus.cc:482] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:30.602159 14236 raft_consensus.cc:504] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.604043 14236 leader_election.cc:288] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037), c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195:37309)
I20260619 14:56:30.612030 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.604625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:56745 (local address 127.11.175.194:42037)
0619 14:56:30.604895 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.604906 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.605800 (+ 894us) server_negotiation.cc:407] Connection header received
0619 14:56:30.606020 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.606025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.606106 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.606248 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:30.606940 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.608057 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.609884 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.611007 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.611135 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.611187 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.611583 (+ 396us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.611676 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 14:56:30.612142 14252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.604305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:42037 (local address 127.11.175.193:56745)
0619 14:56:30.605660 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.605680 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.605870 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.606278 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.606283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.606319 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:30.606760 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.606773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.608226 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.608229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.609722 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.609734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.611230 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.611237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.611412 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.611451 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.611559 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.611708 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1219,"thread_start_us":1047,"threads_started":1}
I20260619 14:56:30.612767 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.604819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:51813 (local address 127.11.175.195:37309)
0619 14:56:30.604994 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.605000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.606452 (+ 1452us) server_negotiation.cc:407] Connection header received
0619 14:56:30.606733 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.606741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.606835 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.606931 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:30.607530 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.608529 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.610569 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.611681 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.611833 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.611907 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.612399 (+ 492us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.612529 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 14:56:30.612850 14251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.604305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:37309 (local address 127.11.175.193:51813)
0619 14:56:30.606348 (+ 2043us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.606368 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.606551 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.606960 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.606968 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.606998 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:30.607338 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.607349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.608724 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.608735 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.610358 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.610371 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.611952 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.611959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.612164 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.612201 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.612371 (+ 170us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.612495 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1812,"thread_start_us":1634,"threads_started":1}
I20260619 14:56:30.612905 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f0ae59b866409b8cdc0fa099bdb001" candidate_uuid: "9b75b22f83244a5093d07fc0d2037b3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" is_pre_election: true
I20260619 14:56:30.613188 14022 raft_consensus.cc:2454] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b75b22f83244a5093d07fc0d2037b3e in term 0.
I20260619 14:56:30.613687 13814 leader_election.cc:302] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ab8b1a7ae8440efaf25ec4a7fad2259, 9b75b22f83244a5093d07fc0d2037b3e; no voters:
I20260619 14:56:30.613747 14162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f0ae59b866409b8cdc0fa099bdb001" candidate_uuid: "9b75b22f83244a5093d07fc0d2037b3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" is_pre_election: true
I20260619 14:56:30.614096 14236 raft_consensus.cc:2781] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:30.614137 14162 raft_consensus.cc:2454] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b75b22f83244a5093d07fc0d2037b3e in term 0.
I20260619 14:56:30.614243 14236 raft_consensus.cc:482] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:30.614358 14236 raft_consensus.cc:3037] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:30.616557 14236 raft_consensus.cc:504] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.617168 14236 leader_election.cc:288] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [CANDIDATE]: Term 1 election: Requested vote from peers 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037), c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195:37309)
I20260619 14:56:30.617583 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f0ae59b866409b8cdc0fa099bdb001" candidate_uuid: "9b75b22f83244a5093d07fc0d2037b3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259"
I20260619 14:56:30.617832 14022 raft_consensus.cc:3037] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:30.617849 14162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f0ae59b866409b8cdc0fa099bdb001" candidate_uuid: "9b75b22f83244a5093d07fc0d2037b3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f"
I20260619 14:56:30.618108 14162 raft_consensus.cc:3037] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:30.620195 14022 raft_consensus.cc:2454] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b75b22f83244a5093d07fc0d2037b3e in term 1.
I20260619 14:56:30.620568 13814 leader_election.cc:302] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ab8b1a7ae8440efaf25ec4a7fad2259, 9b75b22f83244a5093d07fc0d2037b3e; no voters:
I20260619 14:56:30.620572 14162 raft_consensus.cc:2454] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b75b22f83244a5093d07fc0d2037b3e in term 1.
I20260619 14:56:30.620887 14236 raft_consensus.cc:2781] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:30.621366 14236 raft_consensus.cc:686] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 LEADER]: Becoming Leader. State: Replica: 9b75b22f83244a5093d07fc0d2037b3e, State: Running, Role: LEADER
I20260619 14:56:30.621907 14236 consensus_queue.cc:237] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.625403 14253 tablet_replica.cc:439] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: TxnStatusTablet state changed. Reason: New leader 9b75b22f83244a5093d07fc0d2037b3e. Latest consensus state: current_term: 1 leader_uuid: "9b75b22f83244a5093d07fc0d2037b3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } }
I20260619 14:56:30.625943 14253 tablet_replica.cc:442] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: This TxnStatusTablet replica's current role is: LEADER
I20260619 14:56:30.626768 14255 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 14:56:30.627028 14255 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 14:56:30.627158 13724 catalog_manager.cc:5495] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193). New cstate: current_term: 1 leader_uuid: "9b75b22f83244a5093d07fc0d2037b3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } health_report { overall_health: UNKNOWN } } }
I20260619 14:56:30.671404 14247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.665064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38601 (local address 127.0.0.1:40264)
0619 14:56:30.665316 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.665337 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.665468 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.665979 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.665983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.665993 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:56:30.666140 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.666147 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.667623 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.667627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.668639 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.668652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.669645 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.669650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.669801 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.669883 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:56:30.670973 (+ 1090us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:56:30.670991 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.671044 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.671167 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 14:56:30.671496 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.665080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40264 (local address 127.11.175.193:38601)
0619 14:56:30.665271 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.665277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.665421 (+ 144us) server_negotiation.cc:407] Connection header received
0619 14:56:30.665656 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.665661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.665780 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.665948 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:30.666272 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.667491 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.668804 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.669477 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.669599 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.670075 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:30.670868 (+ 793us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:30.671019 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.671087 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.671176 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 14:56:30.679219 14022 raft_consensus.cc:1261] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Refusing update from remote peer 9b75b22f83244a5093d07fc0d2037b3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:30.679241 14162 raft_consensus.cc:1261] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Refusing update from remote peer 9b75b22f83244a5093d07fc0d2037b3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:30.680186 14253 consensus_queue.cc:1035] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:30.680539 14236 consensus_queue.cc:1035] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:30.688601 14238 tablet_replica.cc:439] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: TxnStatusTablet state changed. Reason: New leader 9b75b22f83244a5093d07fc0d2037b3e. Latest consensus state: current_term: 1 leader_uuid: "9b75b22f83244a5093d07fc0d2037b3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } }
I20260619 14:56:30.689041 14238 tablet_replica.cc:442] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:30.694495 14253 tablet_replica.cc:439] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b75b22f83244a5093d07fc0d2037b3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } }
I20260619 14:56:30.694916 14253 tablet_replica.cc:442] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: This TxnStatusTablet replica's current role is: LEADER
I20260619 14:56:30.700788 14237 tablet_replica.cc:439] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: TxnStatusTablet state changed. Reason: New leader 9b75b22f83244a5093d07fc0d2037b3e. Latest consensus state: current_term: 1 leader_uuid: "9b75b22f83244a5093d07fc0d2037b3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } }
I20260619 14:56:30.701104 14237 tablet_replica.cc:442] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:30.701382 14258 mvcc.cc:205] Tried to move back new op lower bound from 7298584537812037632 to 7298584537591332864. Current Snapshot: MvccSnapshot[applied={T|T < 7298584537812037632 or (T in {7298584537812037632})}]
I20260619 14:56:30.703572 14238 tablet_replica.cc:439] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b75b22f83244a5093d07fc0d2037b3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } }
I20260619 14:56:30.703882 14238 tablet_replica.cc:442] T d5f0ae59b866409b8cdc0fa099bdb001 P c4390a811a4a40a4a1bd497ba48a7c6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:30.705440 14236 tablet_replica.cc:439] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b75b22f83244a5093d07fc0d2037b3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } }
I20260619 14:56:30.705808 14236 tablet_replica.cc:442] T d5f0ae59b866409b8cdc0fa099bdb001 P 9b75b22f83244a5093d07fc0d2037b3e: This TxnStatusTablet replica's current role is: LEADER
I20260619 14:56:30.711594 14237 tablet_replica.cc:439] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b75b22f83244a5093d07fc0d2037b3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } } }
I20260619 14:56:30.711867 14237 tablet_replica.cc:442] T d5f0ae59b866409b8cdc0fa099bdb001 P 0ab8b1a7ae8440efaf25ec4a7fad2259: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:56:30.718106 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.709143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.11.175.195:37309)
0619 14:56:30.709311 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.709318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.711373 (+ 2055us) server_negotiation.cc:407] Connection header received
0619 14:56:30.711700 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.711705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.711831 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.711961 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:30.712724 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.713813 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.715898 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.716298 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.716400 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.716905 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:30.717428 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:30.717536 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.717673 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.717741 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 14:56:30.737114 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.729359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:47421 (local address 127.11.175.193:38601)
0619 14:56:30.729626 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.729635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.731353 (+ 1718us) server_negotiation.cc:407] Connection header received
0619 14:56:30.731599 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.731604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.731681 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.731778 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:30.732350 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.733167 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.735248 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.735756 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.735866 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.736431 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:30.736542 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:30.736614 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.736798 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.736872 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 14:56:30.737300 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.729302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38601 (local address 127.11.175.195:47421)
0619 14:56:30.731201 (+ 1899us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.731224 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.731424 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.731822 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.731827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.731842 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:56:30.732163 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.732174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.733304 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.733317 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.735081 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.735099 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.735950 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.735957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.736135 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.736212 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:56:30.736643 (+ 431us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:56:30.736654 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.736740 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.736903 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1711,"thread_start_us":1526,"threads_started":1}
I20260619 14:56:30.748325 14021 raft_consensus.cc:1261] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Refusing update from remote peer c4390a811a4a40a4a1bd497ba48a7c6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:30.748706 13883 raft_consensus.cc:1261] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Refusing update from remote peer c4390a811a4a40a4a1bd497ba48a7c6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:30.749068 14240 consensus_queue.cc:1035] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:30.749456 14238 consensus_queue.cc:1035] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:30.758147 14259 mvcc.cc:205] Tried to move back new op lower bound from 7298584538098520064 to 7298584536555909120. Current Snapshot: MvccSnapshot[applied={T|T < 7298584538098520064}]
I20260619 14:56:30.794936 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.787631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:43883 (local address 127.11.175.195:37309)
0619 14:56:30.787830 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:30.787839 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:30.788465 (+ 626us) server_negotiation.cc:407] Connection header received
0619 14:56:30.788698 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:30.788704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:30.788788 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:30.788876 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:30.789503 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.790666 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.792870 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.793411 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.793574 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.794078 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:30.794221 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:30.794342 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:30.794547 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 14:56:30.794659 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 14:56:30.795725 14276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:30.787576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:37309 (local address 127.11.175.193:43883)
0619 14:56:30.788359 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:30.788379 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:30.788563 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:30.788906 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:30.788910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:30.788926 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:56:30.789330 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.789345 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.790837 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.790843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.792660 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:30.792677 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:30.793618 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:30.793627 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:30.793775 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:30.793873 (+ 98us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:56:30.794396 (+ 523us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:56:30.794409 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 14:56:30.795119 (+ 710us) client_negotiation.cc:240] Negotiation successful
0619 14:56:30.795280 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":447,"threads_started":1}
I20260619 14:56:30.834977 13724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:30.835280 13724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:30.839524 13724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb: Sending DeleteTablet for 3 replicas of tablet d9e794d46a31489ca211398cbe3a49b0
I20260619 14:56:30.840865 14002 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e794d46a31489ca211398cbe3a49b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:30 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:46954
I20260619 14:56:30.840852 14141 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e794d46a31489ca211398cbe3a49b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:30 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:53364
I20260619 14:56:30.841805 14279 tablet_replica.cc:330] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: stopping tablet replica
I20260619 14:56:30.842303 14279 raft_consensus.cc:2229] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:30.842795 14279 raft_consensus.cc:2258] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:30.843046 13724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:56:30.843727 13724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:30.843941 14280 tablet_replica.cc:330] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: stopping tablet replica
I20260619 14:56:30.844422 14280 raft_consensus.cc:2229] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:56:30.845209 14280 raft_consensus.cc:2258] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:30.847326 14280 ts_tablet_manager.cc:1905] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:30.849160 14279 ts_tablet_manager.cc:1905] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:30.852705 14002 tablet_service.cc:1467] Processing CreateTablet for tablet aa85ce6bba6940758f3b8e126b980635 (DEFAULT_TABLE table=test-table [id=a224da57bed54f4e81f33518ac1f09ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:30.853192 14002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa85ce6bba6940758f3b8e126b980635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:30.853721 14140 tablet_service.cc:1467] Processing CreateTablet for tablet aa85ce6bba6940758f3b8e126b980635 (DEFAULT_TABLE table=test-table [id=a224da57bed54f4e81f33518ac1f09ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:30.853760 13862 tablet_service.cc:1467] Processing CreateTablet for tablet aa85ce6bba6940758f3b8e126b980635 (DEFAULT_TABLE table=test-table [id=a224da57bed54f4e81f33518ac1f09ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:30.854336 14140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa85ce6bba6940758f3b8e126b980635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:30.854346 13862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa85ce6bba6940758f3b8e126b980635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:30.858978 13863 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e794d46a31489ca211398cbe3a49b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:30 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:40260
I20260619 14:56:30.861371 14230 tablet_bootstrap.cc:492] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e: Bootstrap starting.
I20260619 14:56:30.861536 14231 tablet_bootstrap.cc:492] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f: Bootstrap starting.
I20260619 14:56:30.863317 14230 tablet_bootstrap.cc:654] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:30.863659 14231 tablet_bootstrap.cc:654] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:30.864275 14233 tablet_bootstrap.cc:492] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Bootstrap starting.
I20260619 14:56:30.866376 14233 tablet_bootstrap.cc:654] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:30.885910 14231 tablet_bootstrap.cc:492] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f: No bootstrap required, opened a new log
I20260619 14:56:30.886234 14231 ts_tablet_manager.cc:1397] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.004s
I20260619 14:56:30.887828 14231 raft_consensus.cc:348] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.888165 14233 tablet_bootstrap.cc:492] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259: No bootstrap required, opened a new log
I20260619 14:56:30.888266 14231 raft_consensus.cc:374] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:30.888378 14233 ts_tablet_manager.cc:1397] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.009s
I20260619 14:56:30.888437 14231 raft_consensus.cc:729] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4390a811a4a40a4a1bd497ba48a7c6f, State: Initialized, Role: FOLLOWER
I20260619 14:56:30.889178 14233 raft_consensus.cc:348] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.889464 14233 raft_consensus.cc:374] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:30.889544 14233 raft_consensus.cc:729] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab8b1a7ae8440efaf25ec4a7fad2259, State: Initialized, Role: FOLLOWER
I20260619 14:56:30.889267 14231 consensus_queue.cc:260] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.889830 14233 consensus_queue.cc:260] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.891115 14233 ts_tablet_manager.cc:1428] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260619 14:56:30.893752 14231 ts_tablet_manager.cc:1428] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260619 14:56:30.896514 14281 tablet_replica.cc:330] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: stopping tablet replica
I20260619 14:56:30.896996 14281 raft_consensus.cc:2229] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:30.897387 14281 raft_consensus.cc:2258] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:30.900933 14279 ts_tablet_manager.cc:1918] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:56:30.903079 14280 ts_tablet_manager.cc:1918] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:56:30.905247 14280 log.cc:1196] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/d9e794d46a31489ca211398cbe3a49b0
I20260619 14:56:30.905642 14281 ts_tablet_manager.cc:1905] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:30.906558 14230 tablet_bootstrap.cc:492] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e: No bootstrap required, opened a new log
I20260619 14:56:30.906858 14230 ts_tablet_manager.cc:1397] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e: Time spent bootstrapping tablet: real 0.046s user 0.020s sys 0.023s
I20260619 14:56:30.907331 14280 ts_tablet_manager.cc:1939] T d9e794d46a31489ca211398cbe3a49b0 P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting consensus metadata
I20260619 14:56:30.907374 14279 log.cc:1196] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/d9e794d46a31489ca211398cbe3a49b0
I20260619 14:56:30.907889 14279 ts_tablet_manager.cc:1939] T d9e794d46a31489ca211398cbe3a49b0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting consensus metadata
I20260619 14:56:30.909278 13712 catalog_manager.cc:4841] TS c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195:37309): tablet d9e794d46a31489ca211398cbe3a49b0 (table test-table [id=a144bf4f62eb471f9870e0ca46a5a799]) successfully deleted
I20260619 14:56:30.909611 13711 catalog_manager.cc:4841] TS 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037): tablet d9e794d46a31489ca211398cbe3a49b0 (table test-table [id=a144bf4f62eb471f9870e0ca46a5a799]) successfully deleted
I20260619 14:56:30.910624 14230 raft_consensus.cc:348] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.911101 14230 raft_consensus.cc:374] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:30.911221 14230 raft_consensus.cc:729] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b75b22f83244a5093d07fc0d2037b3e, State: Initialized, Role: FOLLOWER
I20260619 14:56:30.911549 14230 consensus_queue.cc:260] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:30.912228 14230 ts_tablet_manager.cc:1428] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260619 14:56:30.919575 14281 ts_tablet_manager.cc:1918] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:56:30.919781 14281 log.cc:1196] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/d9e794d46a31489ca211398cbe3a49b0
I20260619 14:56:30.920338 14281 ts_tablet_manager.cc:1939] T d9e794d46a31489ca211398cbe3a49b0 P 9b75b22f83244a5093d07fc0d2037b3e: Deleting consensus metadata
I20260619 14:56:30.922127 13713 catalog_manager.cc:4841] TS 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601): tablet d9e794d46a31489ca211398cbe3a49b0 (table test-table [id=a144bf4f62eb471f9870e0ca46a5a799]) successfully deleted
I20260619 14:56:31.281574 14238 raft_consensus.cc:482] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:31.281819 14238 raft_consensus.cc:504] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:31.282562 14238 leader_election.cc:288] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601), 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037)
I20260619 14:56:31.283300 14021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa85ce6bba6940758f3b8e126b980635" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" is_pre_election: true
I20260619 14:56:31.283308 13883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa85ce6bba6940758f3b8e126b980635" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b75b22f83244a5093d07fc0d2037b3e" is_pre_election: true
I20260619 14:56:31.283567 13883 raft_consensus.cc:2454] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 0.
I20260619 14:56:31.283592 14021 raft_consensus.cc:2454] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 0.
I20260619 14:56:31.284129 14093 leader_election.cc:302] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ab8b1a7ae8440efaf25ec4a7fad2259, c4390a811a4a40a4a1bd497ba48a7c6f; no voters:
I20260619 14:56:31.284466 14238 raft_consensus.cc:2781] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:31.284639 14238 raft_consensus.cc:482] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:31.284747 14238 raft_consensus.cc:3037] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:31.287395 14238 raft_consensus.cc:504] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:31.288092 14238 leader_election.cc:288] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 election: Requested vote from peers 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601), 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037)
I20260619 14:56:31.288556 13883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa85ce6bba6940758f3b8e126b980635" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b75b22f83244a5093d07fc0d2037b3e"
I20260619 14:56:31.288780 13883 raft_consensus.cc:3037] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:31.288906 14021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa85ce6bba6940758f3b8e126b980635" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259"
I20260619 14:56:31.289157 14021 raft_consensus.cc:3037] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:31.291429 14021 raft_consensus.cc:2454] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 1.
I20260619 14:56:31.291771 13883 raft_consensus.cc:2454] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 1.
I20260619 14:56:31.291990 14093 leader_election.cc:302] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ab8b1a7ae8440efaf25ec4a7fad2259, c4390a811a4a40a4a1bd497ba48a7c6f; no voters:
I20260619 14:56:31.292346 14238 raft_consensus.cc:2781] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:31.292511 14238 raft_consensus.cc:686] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 LEADER]: Becoming Leader. State: Replica: c4390a811a4a40a4a1bd497ba48a7c6f, State: Running, Role: LEADER
I20260619 14:56:31.292909 14238 consensus_queue.cc:237] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:31.295933 13725 catalog_manager.cc:5495] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f reported cstate change: term changed from 0 to 1, leader changed from <none> to c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195). New cstate: current_term: 1 leader_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } health_report { overall_health: HEALTHY } } }
I20260619 14:56:31.343750 13883 raft_consensus.cc:1261] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Refusing update from remote peer c4390a811a4a40a4a1bd497ba48a7c6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:31.344374 14274 consensus_queue.cc:1035] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:31.345081 14021 raft_consensus.cc:1261] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Refusing update from remote peer c4390a811a4a40a4a1bd497ba48a7c6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:31.345620 14274 consensus_queue.cc:1035] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 14:56:31.368458 14288 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 14:56:32.963400 13724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:32.963685 13724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:32.968106 13724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb: Sending DeleteTablet for 3 replicas of tablet aa85ce6bba6940758f3b8e126b980635
I20260619 14:56:32.968874 13863 tablet_service.cc:1514] Processing DeleteTablet for tablet aa85ce6bba6940758f3b8e126b980635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:32 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:40260
I20260619 14:56:32.969049 14140 tablet_service.cc:1514] Processing DeleteTablet for tablet aa85ce6bba6940758f3b8e126b980635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:32 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:53364
I20260619 14:56:32.969070 14002 tablet_service.cc:1514] Processing DeleteTablet for tablet aa85ce6bba6940758f3b8e126b980635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:32 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:46954
I20260619 14:56:32.981056 14321 tablet_replica.cc:330] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e: stopping tablet replica
I20260619 14:56:32.981531 14321 raft_consensus.cc:2229] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:32.981930 14321 raft_consensus.cc:2258] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:32.983223 14321 ts_tablet_manager.cc:1905] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:32.986238 14322 tablet_replica.cc:330] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f: stopping tablet replica
I20260619 14:56:32.986486 14323 tablet_replica.cc:330] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259: stopping tablet replica
I20260619 14:56:32.986479 14322 raft_consensus.cc:2229] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:56:32.986725 14323 raft_consensus.cc:2229] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:32.986951 14322 raft_consensus.cc:2258] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:32.986999 14323 raft_consensus.cc:2258] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:32.987955 14323 ts_tablet_manager.cc:1905] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:32.988018 14322 ts_tablet_manager.cc:1905] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:32.991590 14321 ts_tablet_manager.cc:1918] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 14:56:32.991797 14321 log.cc:1196] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/aa85ce6bba6940758f3b8e126b980635
I20260619 14:56:32.992332 14321 ts_tablet_manager.cc:1939] T aa85ce6bba6940758f3b8e126b980635 P 9b75b22f83244a5093d07fc0d2037b3e: Deleting consensus metadata
I20260619 14:56:32.993814 13713 catalog_manager.cc:4841] TS 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601): tablet aa85ce6bba6940758f3b8e126b980635 (table test-table [id=a224da57bed54f4e81f33518ac1f09ab]) successfully deleted
I20260619 14:56:32.994432 14323 ts_tablet_manager.cc:1918] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 14:56:32.994571 14323 log.cc:1196] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/aa85ce6bba6940758f3b8e126b980635
I20260619 14:56:32.994997 14322 ts_tablet_manager.cc:1918] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 14:56:32.995105 14323 ts_tablet_manager.cc:1939] T aa85ce6bba6940758f3b8e126b980635 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting consensus metadata
I20260619 14:56:32.995148 14322 log.cc:1196] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/aa85ce6bba6940758f3b8e126b980635
I20260619 14:56:32.995609 14322 ts_tablet_manager.cc:1939] T aa85ce6bba6940758f3b8e126b980635 P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting consensus metadata
I20260619 14:56:32.996163 13711 catalog_manager.cc:4841] TS 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037): tablet aa85ce6bba6940758f3b8e126b980635 (table test-table [id=a224da57bed54f4e81f33518ac1f09ab]) successfully deleted
I20260619 14:56:32.996697 13712 catalog_manager.cc:4841] TS c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195:37309): tablet aa85ce6bba6940758f3b8e126b980635 (table test-table [id=a224da57bed54f4e81f33518ac1f09ab]) successfully deleted
Jun 19 14:56:33 dist-test-slave-76rk krb5kdc[13681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 14:56:33 dist-test-slave-76rk krb5kdc[13681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:33.039233 14331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.017085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.11.175.254:45347)
0619 14:56:33.017687 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:33.017692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:33.017713 (+ 21us) server_negotiation.cc:407] Connection header received
0619 14:56:33.017879 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:33.017883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:33.017947 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:33.018040 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:33.019476 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.020321 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.021561 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.022026 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.022125 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.033353 (+ 11228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:33.033360 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:33.033364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:33.033407 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:33.036200 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:33.036999 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:33.037004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:33.037006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:33.037143 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:33.037575 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:33.037579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:33.037581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:33.037895 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:33.038088 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:33.038557 (+ 469us) server_negotiation.cc:299] Negotiation successful
0619 14:56:33.038845 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":327,"threads_started":1}
I20260619 14:56:33.043038 13724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:56:33.043668 13724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:33.050119 13863 tablet_service.cc:1467] Processing CreateTablet for tablet 58b847114f904690b0df53db75fe2b5b (DEFAULT_TABLE table=test-table [id=7bfbc962f32c41c181ac990d13b1cc99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:33.050365 14002 tablet_service.cc:1467] Processing CreateTablet for tablet 58b847114f904690b0df53db75fe2b5b (DEFAULT_TABLE table=test-table [id=7bfbc962f32c41c181ac990d13b1cc99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:33.050606 13863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b847114f904690b0df53db75fe2b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:33.050485 14140 tablet_service.cc:1467] Processing CreateTablet for tablet 58b847114f904690b0df53db75fe2b5b (DEFAULT_TABLE table=test-table [id=7bfbc962f32c41c181ac990d13b1cc99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:33.050802 14002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b847114f904690b0df53db75fe2b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:33.051002 14140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b847114f904690b0df53db75fe2b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:33.056110 14332 tablet_bootstrap.cc:492] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f: Bootstrap starting.
I20260619 14:56:33.057760 14334 tablet_bootstrap.cc:492] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259: Bootstrap starting.
I20260619 14:56:33.058094 14332 tablet_bootstrap.cc:654] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:33.059751 14334 tablet_bootstrap.cc:654] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:33.060256 14333 tablet_bootstrap.cc:492] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e: Bootstrap starting.
I20260619 14:56:33.061775 14332 tablet_bootstrap.cc:492] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f: No bootstrap required, opened a new log
I20260619 14:56:33.061962 14332 ts_tablet_manager.cc:1397] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260619 14:56:33.061982 14333 tablet_bootstrap.cc:654] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:33.062623 14334 tablet_bootstrap.cc:492] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259: No bootstrap required, opened a new log
I20260619 14:56:33.062889 14334 ts_tablet_manager.cc:1397] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 14:56:33.062834 14332 raft_consensus.cc:348] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.063148 14332 raft_consensus.cc:374] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:33.063275 14332 raft_consensus.cc:729] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4390a811a4a40a4a1bd497ba48a7c6f, State: Initialized, Role: FOLLOWER
I20260619 14:56:33.063520 14332 consensus_queue.cc:260] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.064098 14332 ts_tablet_manager.cc:1428] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 14:56:33.063928 14334 raft_consensus.cc:348] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.064198 14334 raft_consensus.cc:374] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:33.064272 14333 tablet_bootstrap.cc:492] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e: No bootstrap required, opened a new log
I20260619 14:56:33.064307 14334 raft_consensus.cc:729] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab8b1a7ae8440efaf25ec4a7fad2259, State: Initialized, Role: FOLLOWER
I20260619 14:56:33.064465 14333 ts_tablet_manager.cc:1397] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260619 14:56:33.064550 14334 consensus_queue.cc:260] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.065613 14334 ts_tablet_manager.cc:1428] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 14:56:33.065299 14333 raft_consensus.cc:348] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.066176 14333 raft_consensus.cc:374] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:33.066298 14333 raft_consensus.cc:729] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b75b22f83244a5093d07fc0d2037b3e, State: Initialized, Role: FOLLOWER
I20260619 14:56:33.066556 14333 consensus_queue.cc:260] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.067195 14333 ts_tablet_manager.cc:1428] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 14:56:33.139779 14299 raft_consensus.cc:482] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:33.140009 14299 raft_consensus.cc:504] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.140915 14299 leader_election.cc:288] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601), 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037)
I20260619 14:56:33.141459 14021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b847114f904690b0df53db75fe2b5b" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" is_pre_election: true
I20260619 14:56:33.141551 13883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b847114f904690b0df53db75fe2b5b" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b75b22f83244a5093d07fc0d2037b3e" is_pre_election: true
I20260619 14:56:33.141738 14021 raft_consensus.cc:2454] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 0.
I20260619 14:56:33.141880 13883 raft_consensus.cc:2454] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 0.
I20260619 14:56:33.142309 14093 leader_election.cc:302] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ab8b1a7ae8440efaf25ec4a7fad2259, c4390a811a4a40a4a1bd497ba48a7c6f; no voters:
I20260619 14:56:33.142712 14299 raft_consensus.cc:2781] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:33.142858 14299 raft_consensus.cc:482] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:33.142961 14299 raft_consensus.cc:3037] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:33.145221 14299 raft_consensus.cc:504] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.145895 14299 leader_election.cc:288] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 election: Requested vote from peers 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601), 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037)
I20260619 14:56:33.146472 14021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b847114f904690b0df53db75fe2b5b" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259"
I20260619 14:56:33.146466 13883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b847114f904690b0df53db75fe2b5b" candidate_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b75b22f83244a5093d07fc0d2037b3e"
I20260619 14:56:33.146725 14021 raft_consensus.cc:3037] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:33.146706 13883 raft_consensus.cc:3037] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:33.149682 14021 raft_consensus.cc:2454] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 1.
I20260619 14:56:33.149689 13883 raft_consensus.cc:2454] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4390a811a4a40a4a1bd497ba48a7c6f in term 1.
I20260619 14:56:33.150197 14095 leader_election.cc:302] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b75b22f83244a5093d07fc0d2037b3e, c4390a811a4a40a4a1bd497ba48a7c6f; no voters:
I20260619 14:56:33.150545 14299 raft_consensus.cc:2781] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:33.150735 14299 raft_consensus.cc:686] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 LEADER]: Becoming Leader. State: Replica: c4390a811a4a40a4a1bd497ba48a7c6f, State: Running, Role: LEADER
I20260619 14:56:33.151016 14299 consensus_queue.cc:237] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.154318 13725 catalog_manager.cc:5495] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f reported cstate change: term changed from 0 to 1, leader changed from <none> to c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195). New cstate: current_term: 1 leader_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } health_report { overall_health: HEALTHY } } }
I20260619 14:56:33.176831 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.171834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60154 (local address 127.11.175.195:37309)
0619 14:56:33.172354 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:33.172359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:33.172376 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:56:33.172427 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:33.172430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:33.172488 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:33.172569 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:33.172945 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.173799 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.175088 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.175517 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.175609 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.176044 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:33.176152 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:33.176221 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:33.176403 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 14:56:33.176463 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":287,"threads_started":1}
I20260619 14:56:33.185792 14021 raft_consensus.cc:1261] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Refusing update from remote peer c4390a811a4a40a4a1bd497ba48a7c6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:33.185938 13883 raft_consensus.cc:1261] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Refusing update from remote peer c4390a811a4a40a4a1bd497ba48a7c6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:33.186434 14316 consensus_queue.cc:1035] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:33.186738 14299 consensus_queue.cc:1035] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:33.229877 13724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:33.230146 13724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:33.233194 13724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb: Sending DeleteTablet for 3 replicas of tablet 58b847114f904690b0df53db75fe2b5b
I20260619 14:56:33.234108 13863 tablet_service.cc:1514] Processing DeleteTablet for tablet 58b847114f904690b0df53db75fe2b5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:33 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:40260
I20260619 14:56:33.234177 14140 tablet_service.cc:1514] Processing DeleteTablet for tablet 58b847114f904690b0df53db75fe2b5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:33 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:53364
I20260619 14:56:33.234193 14002 tablet_service.cc:1514] Processing DeleteTablet for tablet 58b847114f904690b0df53db75fe2b5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:33 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:46954
I20260619 14:56:33.234475 14323 tablet_replica.cc:330] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259: stopping tablet replica
I20260619 14:56:33.234473 14322 tablet_replica.cc:330] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f: stopping tablet replica
I20260619 14:56:33.234712 14323 raft_consensus.cc:2229] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:33.234735 14322 raft_consensus.cc:2229] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:56:33.234810 14321 tablet_replica.cc:330] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e: stopping tablet replica
I20260619 14:56:33.234925 14323 raft_consensus.cc:2258] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:33.234989 14321 raft_consensus.cc:2229] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:33.235075 14322 raft_consensus.cc:2258] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:33.235170 14321 raft_consensus.cc:2258] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:33.235670 13724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 14:56:33.236040 14323 ts_tablet_manager.cc:1905] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 14:56:33.236337 13724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:33.236514 14321 ts_tablet_manager.cc:1905] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:33.238270 14322 ts_tablet_manager.cc:1905] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:33.241811 14323 ts_tablet_manager.cc:1918] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:56:33.241990 14323 log.cc:1196] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/58b847114f904690b0df53db75fe2b5b
I20260619 14:56:33.242419 14323 ts_tablet_manager.cc:1939] T 58b847114f904690b0df53db75fe2b5b P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting consensus metadata
I20260619 14:56:33.243628 13711 catalog_manager.cc:4841] TS 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037): tablet 58b847114f904690b0df53db75fe2b5b (table test-table [id=7bfbc962f32c41c181ac990d13b1cc99]) successfully deleted
I20260619 14:56:33.243974 14321 ts_tablet_manager.cc:1918] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:56:33.244127 14321 log.cc:1196] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/58b847114f904690b0df53db75fe2b5b
I20260619 14:56:33.244542 14321 ts_tablet_manager.cc:1939] T 58b847114f904690b0df53db75fe2b5b P 9b75b22f83244a5093d07fc0d2037b3e: Deleting consensus metadata
I20260619 14:56:33.245599 13713 catalog_manager.cc:4841] TS 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601): tablet 58b847114f904690b0df53db75fe2b5b (table test-table [id=7bfbc962f32c41c181ac990d13b1cc99]) successfully deleted
I20260619 14:56:33.246016 14322 ts_tablet_manager.cc:1918] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:56:33.246179 14322 log.cc:1196] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/58b847114f904690b0df53db75fe2b5b
I20260619 14:56:33.246690 14322 ts_tablet_manager.cc:1939] T 58b847114f904690b0df53db75fe2b5b P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting consensus metadata
I20260619 14:56:33.246954 14002 tablet_service.cc:1467] Processing CreateTablet for tablet 7980e13fa924464c909677cdea5c0fa0 (DEFAULT_TABLE table=test-table [id=ee632e44dce44ce18a53a5aa22a236b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:33.247037 14140 tablet_service.cc:1467] Processing CreateTablet for tablet 7980e13fa924464c909677cdea5c0fa0 (DEFAULT_TABLE table=test-table [id=ee632e44dce44ce18a53a5aa22a236b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:33.247298 13863 tablet_service.cc:1467] Processing CreateTablet for tablet 7980e13fa924464c909677cdea5c0fa0 (DEFAULT_TABLE table=test-table [id=ee632e44dce44ce18a53a5aa22a236b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:33.247422 14002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7980e13fa924464c909677cdea5c0fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:33.247689 13863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7980e13fa924464c909677cdea5c0fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:33.248003 14140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7980e13fa924464c909677cdea5c0fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:33.252457 14333 tablet_bootstrap.cc:492] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e: Bootstrap starting.
I20260619 14:56:33.253749 14334 tablet_bootstrap.cc:492] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Bootstrap starting.
I20260619 14:56:33.254249 14333 tablet_bootstrap.cc:654] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:33.255105 14332 tablet_bootstrap.cc:492] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f: Bootstrap starting.
I20260619 14:56:33.255399 14334 tablet_bootstrap.cc:654] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:33.256466 13712 catalog_manager.cc:4841] TS c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195:37309): tablet 58b847114f904690b0df53db75fe2b5b (table test-table [id=7bfbc962f32c41c181ac990d13b1cc99]) successfully deleted
I20260619 14:56:33.257107 14332 tablet_bootstrap.cc:654] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:33.258174 14333 tablet_bootstrap.cc:492] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e: No bootstrap required, opened a new log
I20260619 14:56:33.258402 14333 ts_tablet_manager.cc:1397] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.001s
I20260619 14:56:33.259155 14334 tablet_bootstrap.cc:492] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: No bootstrap required, opened a new log
I20260619 14:56:33.259348 14334 ts_tablet_manager.cc:1397] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 14:56:33.259282 14333 raft_consensus.cc:348] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.259562 14333 raft_consensus.cc:374] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:33.259653 14333 raft_consensus.cc:729] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b75b22f83244a5093d07fc0d2037b3e, State: Initialized, Role: FOLLOWER
I20260619 14:56:33.259905 14333 consensus_queue.cc:260] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.260080 14332 tablet_bootstrap.cc:492] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f: No bootstrap required, opened a new log
I20260619 14:56:33.260246 14332 ts_tablet_manager.cc:1397] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260619 14:56:33.260123 14334 raft_consensus.cc:348] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.260367 14334 raft_consensus.cc:374] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:33.260450 14334 raft_consensus.cc:729] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab8b1a7ae8440efaf25ec4a7fad2259, State: Initialized, Role: FOLLOWER
I20260619 14:56:33.260533 14333 ts_tablet_manager.cc:1428] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 14:56:33.260681 14334 consensus_queue.cc:260] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.261234 14334 ts_tablet_manager.cc:1428] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 14:56:33.261001 14332 raft_consensus.cc:348] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.262172 14332 raft_consensus.cc:374] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:33.262275 14332 raft_consensus.cc:729] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4390a811a4a40a4a1bd497ba48a7c6f, State: Initialized, Role: FOLLOWER
I20260619 14:56:33.262516 14332 consensus_queue.cc:260] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.263152 14332 ts_tablet_manager.cc:1428] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 14:56:33.474586 14338 raft_consensus.cc:482] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:33.474932 14338 raft_consensus.cc:504] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.476497 14338 leader_election.cc:288] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601), c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195:37309)
I20260619 14:56:33.486290 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.476970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:38953 (local address 127.11.175.195:37309)
0619 14:56:33.477176 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:33.477185 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:33.478136 (+ 951us) server_negotiation.cc:407] Connection header received
0619 14:56:33.478372 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:33.478378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:33.478459 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:33.478567 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:33.479467 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.480666 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.483597 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.484938 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.485123 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.485202 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:33.485891 (+ 689us) server_negotiation.cc:299] Negotiation successful
0619 14:56:33.486021 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 14:56:33.486377 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.476970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:46217 (local address 127.11.175.193:38601)
0619 14:56:33.477669 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:33.477676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:33.478240 (+ 564us) server_negotiation.cc:407] Connection header received
0619 14:56:33.478371 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:33.478377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:33.478459 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:33.478567 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:33.479421 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.480666 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.483597 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.484927 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.485122 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.485202 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:33.485860 (+ 658us) server_negotiation.cc:299] Negotiation successful
0619 14:56:33.486007 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":391,"threads_started":1}
I20260619 14:56:33.486558 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.476896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:37309 (local address 127.11.175.194:38953)
0619 14:56:33.478001 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:33.478021 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:33.478203 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:33.478608 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:33.478613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:33.478699 (+ 86us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:33.479240 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:33.479252 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.480825 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.480830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:33.483383 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:33.483402 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.485247 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.485254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:33.485609 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.485692 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 14:56:33.485833 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 14:56:33.486031 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"mutex_wait_us":300,"spinlock_wait_cycles":7552,"thread_start_us":949,"threads_started":1}
I20260619 14:56:33.486605 14349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.476945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38601 (local address 127.11.175.194:46217)
0619 14:56:33.478139 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:33.478155 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:33.478256 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:33.478608 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:33.478613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:33.478662 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:33.479148 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:33.479169 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.480859 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.480864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:33.483384 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:33.483402 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.485255 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.485261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:33.485605 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.485692 (+ 87us) client_negotiation.cc:769] Sending connection context
0619 14:56:33.485795 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 14:56:33.486031 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"mutex_wait_us":214,"spinlock_wait_cycles":1408,"thread_start_us":653,"threads_started":1}
I20260619 14:56:33.487524 13883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7980e13fa924464c909677cdea5c0fa0" candidate_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b75b22f83244a5093d07fc0d2037b3e" is_pre_election: true
I20260619 14:56:33.487800 13883 raft_consensus.cc:2454] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ab8b1a7ae8440efaf25ec4a7fad2259 in term 0.
I20260619 14:56:33.487723 14162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7980e13fa924464c909677cdea5c0fa0" candidate_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" is_pre_election: true
I20260619 14:56:33.488006 14162 raft_consensus.cc:2454] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ab8b1a7ae8440efaf25ec4a7fad2259 in term 0.
I20260619 14:56:33.488540 13956 leader_election.cc:302] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ab8b1a7ae8440efaf25ec4a7fad2259, 9b75b22f83244a5093d07fc0d2037b3e; no voters:
I20260619 14:56:33.488963 14338 raft_consensus.cc:2781] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:33.489078 14338 raft_consensus.cc:482] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:33.489198 14338 raft_consensus.cc:3037] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:33.491659 14338 raft_consensus.cc:504] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.492416 14338 leader_election.cc:288] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [CANDIDATE]: Term 1 election: Requested vote from peers 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601), c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195:37309)
I20260619 14:56:33.492918 13883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7980e13fa924464c909677cdea5c0fa0" candidate_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b75b22f83244a5093d07fc0d2037b3e"
I20260619 14:56:33.493050 14162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7980e13fa924464c909677cdea5c0fa0" candidate_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f"
I20260619 14:56:33.493158 13883 raft_consensus.cc:3037] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:33.493271 14162 raft_consensus.cc:3037] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:33.496228 13883 raft_consensus.cc:2454] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ab8b1a7ae8440efaf25ec4a7fad2259 in term 1.
I20260619 14:56:33.496233 14162 raft_consensus.cc:2454] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ab8b1a7ae8440efaf25ec4a7fad2259 in term 1.
I20260619 14:56:33.496779 13955 leader_election.cc:302] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0ab8b1a7ae8440efaf25ec4a7fad2259, 9b75b22f83244a5093d07fc0d2037b3e, c4390a811a4a40a4a1bd497ba48a7c6f; no voters:
I20260619 14:56:33.497177 14338 raft_consensus.cc:2781] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:33.497663 14338 raft_consensus.cc:686] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 LEADER]: Becoming Leader. State: Replica: 0ab8b1a7ae8440efaf25ec4a7fad2259, State: Running, Role: LEADER
I20260619 14:56:33.498108 14338 consensus_queue.cc:237] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } }
I20260619 14:56:33.501765 13724 catalog_manager.cc:5495] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194). New cstate: current_term: 1 leader_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ab8b1a7ae8440efaf25ec4a7fad2259" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 42037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 } health_report { overall_health: UNKNOWN } } }
I20260619 14:56:33.568295 14354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.562061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.11.175.194:42037)
0619 14:56:33.562720 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:33.562729 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:33.562758 (+ 29us) server_negotiation.cc:407] Connection header received
0619 14:56:33.562840 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:33.562844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:33.562919 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:33.563069 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:33.563468 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.564415 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.565825 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.566237 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.566366 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.566997 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:33.567595 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:33.567760 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:33.567847 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 14:56:33.567931 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":348,"threads_started":1}
I20260619 14:56:33.584894 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.578804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:54435 (local address 127.11.175.193:38601)
0619 14:56:33.579003 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:33.579008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:33.579605 (+ 597us) server_negotiation.cc:407] Connection header received
0619 14:56:33.579800 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:33.579804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:33.579878 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:33.579965 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:33.580422 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.581279 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.582949 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.583407 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.583582 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.584060 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:33.584186 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:33.584296 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:33.584461 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 14:56:33.584551 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 14:56:33.585006 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.578777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38601 (local address 127.11.175.194:54435)
0619 14:56:33.579500 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:33.579517 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:33.579672 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:33.579988 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:33.579991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:33.580001 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:56:33.580263 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:33.580273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.581437 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.581443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:33.582789 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:33.582803 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.583618 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.583624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:33.583790 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.583851 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:56:33.584296 (+ 445us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:56:33.584313 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 14:56:33.584425 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 14:56:33.584589 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":375,"threads_started":1}
I20260619 14:56:33.593796 14162 raft_consensus.cc:1261] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Refusing update from remote peer 0ab8b1a7ae8440efaf25ec4a7fad2259: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:33.594456 14352 consensus_queue.cc:1035] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4390a811a4a40a4a1bd497ba48a7c6f" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 37309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:33.598912 14304 mvcc.cc:205] Tried to move back new op lower bound from 7298584549754355712 to 7298584549372162048. Current Snapshot: MvccSnapshot[applied={T|T < 7298584549754355712}]
I20260619 14:56:33.599087 13883 raft_consensus.cc:1261] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Refusing update from remote peer 0ab8b1a7ae8440efaf25ec4a7fad2259: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:33.599980 14338 consensus_queue.cc:1035] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b75b22f83244a5093d07fc0d2037b3e" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:33.627562 14354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.620765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:60497 (local address 127.11.175.194:42037)
0619 14:56:33.620961 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:33.620969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:33.621645 (+ 676us) server_negotiation.cc:407] Connection header received
0619 14:56:33.621872 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:33.621880 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:33.621968 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:33.622077 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:33.622578 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.623818 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.625643 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.626130 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.626285 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.626777 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:33.626889 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:33.627008 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:33.627175 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 14:56:33.627260 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 14:56:33.627774 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:33.620803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:42037 (local address 127.11.175.193:60497)
0619 14:56:33.621532 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:33.621550 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:33.621720 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:33.622093 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:33.622098 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:33.622110 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:56:33.622423 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:33.622436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.623952 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.623958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:33.625484 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:33.625500 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:33.626319 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:33.626326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:33.626491 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:33.626531 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:56:33.627010 (+ 479us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:56:33.627025 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 14:56:33.627135 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 14:56:33.627299 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":416,"threads_started":1}
W20260619 14:56:33.628899 14304 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 14:56:35.232868 13724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:35.233117 13724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:35.237531 13724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1657435936bf4af38823b5165ace53bb: Sending DeleteTablet for 3 replicas of tablet 7980e13fa924464c909677cdea5c0fa0
I20260619 14:56:35.238575 13863 tablet_service.cc:1514] Processing DeleteTablet for tablet 7980e13fa924464c909677cdea5c0fa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:35 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:40260
I20260619 14:56:35.238576 14002 tablet_service.cc:1514] Processing DeleteTablet for tablet 7980e13fa924464c909677cdea5c0fa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:35 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:46954
I20260619 14:56:35.238709 14140 tablet_service.cc:1514] Processing DeleteTablet for tablet 7980e13fa924464c909677cdea5c0fa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:35 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:53364
I20260619 14:56:35.239511 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13797
I20260619 14:56:35.248077 14388 tablet_replica.cc:330] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f: stopping tablet replica
I20260619 14:56:35.248258 14386 tablet_replica.cc:330] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: stopping tablet replica
I20260619 14:56:35.248347 14388 raft_consensus.cc:2229] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:35.248463 14386 raft_consensus.cc:2229] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:56:35.248507 14387 tablet_replica.cc:330] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e: stopping tablet replica
I20260619 14:56:35.248726 14387 raft_consensus.cc:2229] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:35.248989 14387 raft_consensus.cc:2258] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:35.249054 14386 raft_consensus.cc:2258] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:35.249734 14388 raft_consensus.cc:2258] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:35.250113 14387 ts_tablet_manager.cc:1905] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:35.250597 14386 ts_tablet_manager.cc:1905] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:35.250906 14388 ts_tablet_manager.cc:1905] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 14:56:35 dist-test-slave-76rk krb5kdc[13681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:56:35.257073 14387 ts_tablet_manager.cc:1918] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 14:56:35.257218 14387 log.cc:1196] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/7980e13fa924464c909677cdea5c0fa0
I20260619 14:56:35.257752 14388 ts_tablet_manager.cc:1918] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 14:56:35.257766 14386 ts_tablet_manager.cc:1918] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 14:56:35.257799 14387 ts_tablet_manager.cc:1939] T 7980e13fa924464c909677cdea5c0fa0 P 9b75b22f83244a5093d07fc0d2037b3e: Deleting consensus metadata
I20260619 14:56:35.257900 14386 log.cc:1196] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/7980e13fa924464c909677cdea5c0fa0
I20260619 14:56:35.258050 14388 log.cc:1196] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/7980e13fa924464c909677cdea5c0fa0
I20260619 14:56:35.258368 14386 ts_tablet_manager.cc:1939] T 7980e13fa924464c909677cdea5c0fa0 P 0ab8b1a7ae8440efaf25ec4a7fad2259: Deleting consensus metadata
I20260619 14:56:35.258594 14388 ts_tablet_manager.cc:1939] T 7980e13fa924464c909677cdea5c0fa0 P c4390a811a4a40a4a1bd497ba48a7c6f: Deleting consensus metadata
I20260619 14:56:35.259156 13713 catalog_manager.cc:4841] TS 9b75b22f83244a5093d07fc0d2037b3e (127.11.175.193:38601): tablet 7980e13fa924464c909677cdea5c0fa0 (table test-table [id=ee632e44dce44ce18a53a5aa22a236b8]) successfully deleted
I20260619 14:56:35.259586 13711 catalog_manager.cc:4841] TS 0ab8b1a7ae8440efaf25ec4a7fad2259 (127.11.175.194:42037): tablet 7980e13fa924464c909677cdea5c0fa0 (table test-table [id=ee632e44dce44ce18a53a5aa22a236b8]) successfully deleted
I20260619 14:56:35.259790 13712 catalog_manager.cc:4841] TS c4390a811a4a40a4a1bd497ba48a7c6f (127.11.175.195:37309): tablet 7980e13fa924464c909677cdea5c0fa0 (table test-table [id=ee632e44dce44ce18a53a5aa22a236b8]) successfully deleted
I20260619 14:56:35.264386 14390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:35.240883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46582 (local address 127.11.175.193:38601)
0619 14:56:35.242895 (+ 2012us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:35.242904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:35.242936 (+ 32us) server_negotiation.cc:407] Connection header received
0619 14:56:35.243015 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:35.243024 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:35.243125 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:35.245618 (+ 2493us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:35.245874 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:35.247444 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:35.251168 (+ 3724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:35.251834 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:35.251962 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:35.257726 (+ 5764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:35.257751 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:35.257771 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:35.257830 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:35.261441 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:35.262159 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:35.262170 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:35.262173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:35.262309 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:35.262788 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:35.262791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:35.262793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:35.263255 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:35.263488 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:35.263824 (+ 336us) server_negotiation.cc:299] Negotiation successful
0619 14:56:35.264038 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1825,"thread_start_us":1599,"threads_started":1}
W20260619 14:56:35.265229 13928 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46582
W20260619 14:56:35.265882 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13797: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 14:56:35.266136 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 13797
I20260619 14:56:35.289642 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13937
Jun 19 14:56:35 dist-test-slave-76rk krb5kdc[13681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:56:35.306098 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:35.290429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41282 (local address 127.11.175.194:42037)
0619 14:56:35.291078 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:35.291082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:35.291098 (+ 16us) server_negotiation.cc:407] Connection header received
0619 14:56:35.291152 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:35.291156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:35.291219 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:35.291312 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:35.292704 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:35.293434 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:35.294916 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:35.295320 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:35.295451 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:35.300076 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:35.300090 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:35.300105 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:35.300142 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:35.303256 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:35.303950 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:35.303958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:35.303960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:35.304099 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:35.304429 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:35.304432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:35.304434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:35.304771 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:35.305076 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:35.305435 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 14:56:35.305687 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":370,"threads_started":1}
W20260619 14:56:35.306942 14067 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41282
W20260619 14:56:35.307627 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13937: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 14:56:35.307761 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 13937
I20260619 14:56:35.330787 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 14076
Jun 19 14:56:35 dist-test-slave-76rk krb5kdc[13681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:56:35.346539 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:35.331606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60164 (local address 127.11.175.195:37309)
0619 14:56:35.332115 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:35.332120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:35.332136 (+ 16us) server_negotiation.cc:407] Connection header received
0619 14:56:35.332190 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:35.332194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:35.332252 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:35.332347 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:35.333678 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:35.334399 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:35.335793 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:35.336189 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:35.336292 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:35.340748 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:35.340764 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:35.340779 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:35.340822 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:35.343741 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:35.344385 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:35.344393 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:35.344395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:35.344513 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:35.344956 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:35.344961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:35.344963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:35.345374 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:35.345610 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:35.345952 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 14:56:35.346136 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":272,"threads_started":1}
W20260619 14:56:35.347352 14206 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60164
W20260619 14:56:35.348150 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 14076: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 14:56:35.348351 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 14076
I20260619 14:56:35.372519 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 13694
I20260619 14:56:35.373947 13766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:35.493119 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 13694
2026-06-19T14:56:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8846 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:56:35 dist-test-slave-76rk krb5kdc[14396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:56:35 dist-test-slave-76rk krb5kdc[14396](info): set up 2 sockets
Jun 19 14:56:35 dist-test-slave-76rk krb5kdc[14396](info): commencing operation
krb5kdc: starting...
W20260619 14:56:36.606989 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:56:36 dist-test-slave-76rk krb5kdc[14396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:56:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:56:36.807370 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41631
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:42013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:41631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:37.144510 14409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:37.144906 14409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:37.145071 14409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:37.152601 14409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:37.152792 14409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:37.152902 14409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:37.153005 14409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:37.153082 14409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:56:37.163228 14409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:42013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:41631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:37.164574 14409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:37.165899 14409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:37.178506 14417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:37.178707 14409 server_base.cc:1034] running on GCE node
W20260619 14:56:37.178414 14415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:37.178421 14414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:37.179883 14409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:37.181303 14409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:37.182592 14409 hybrid_clock.cc:648] HybridClock initialized: now 1781880997182508 us; error 65 us; skew 500 ppm
I20260619 14:56:37.183074 14409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:37 dist-test-slave-76rk krb5kdc[14396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880997, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:37.186606 14409 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:56:37.188308 14409 webserver.cc:460] Webserver started at http://127.11.175.254:38683/ using document root <none> and password file <none>
I20260619 14:56:37.189082 14409 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:37.189241 14409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:37.189591 14409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:37.192848 14409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "dd3f390796e0440e95dab758b0254e67"
format_stamp: "Formatted at 2026-06-19 14:56:37 on dist-test-slave-76rk"
server_key: "44d4ba3a2a7566b7be4d8b55bc65cc4f"
server_key_iv: "5e3bdc30cfc6aaeeea6c865411ba3835"
server_key_version: "encryptionkey@0"
I20260619 14:56:37.193730 14409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "dd3f390796e0440e95dab758b0254e67"
format_stamp: "Formatted at 2026-06-19 14:56:37 on dist-test-slave-76rk"
server_key: "44d4ba3a2a7566b7be4d8b55bc65cc4f"
server_key_iv: "5e3bdc30cfc6aaeeea6c865411ba3835"
server_key_version: "encryptionkey@0"
I20260619 14:56:37.199466 14409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 14:56:37.203568 14424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:37.205191 14409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:37.205529 14409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "dd3f390796e0440e95dab758b0254e67"
format_stamp: "Formatted at 2026-06-19 14:56:37 on dist-test-slave-76rk"
server_key: "44d4ba3a2a7566b7be4d8b55bc65cc4f"
server_key_iv: "5e3bdc30cfc6aaeeea6c865411ba3835"
server_key_version: "encryptionkey@0"
I20260619 14:56:37.205756 14409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:37.242274 14409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:37.247037 14409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:37.247380 14409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:37.266399 14409 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:41631
I20260619 14:56:37.266431 14475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:41631 every 8 connection(s)
I20260619 14:56:37.267907 14409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:37.271955 14476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:37.278244 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 14409
I20260619 14:56:37.278502 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:37.279229 11967 external_mini_cluster.cc:1428] Setting key 6efe9010005f4c9d9467a17f964fe665
I20260619 14:56:37.283429 14476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67: Bootstrap starting.
I20260619 14:56:37.287465 14476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:37.288697 14476 log.cc:824] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:37.292157 14476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67: No bootstrap required, opened a new log
Jun 19 14:56:37 dist-test-slave-76rk krb5kdc[14396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:37.300981 14476 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f390796e0440e95dab758b0254e67" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41631 } }
I20260619 14:56:37.301432 14476 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:37.301571 14476 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd3f390796e0440e95dab758b0254e67, State: Initialized, Role: FOLLOWER
I20260619 14:56:37.302273 14476 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f390796e0440e95dab758b0254e67" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41631 } }
I20260619 14:56:37.302574 14476 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:37.302731 14476 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:37.302942 14476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:37.303797 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:37.281237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.11.175.254:41631)
0619 14:56:37.282074 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:37.282085 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:37.282133 (+ 48us) server_negotiation.cc:407] Connection header received
0619 14:56:37.284039 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:37.284064 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:37.284476 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:37.284825 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:37.286787 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:37.288220 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:37.289949 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:37.290744 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:37.290880 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:37.295594 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:37.295602 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:37.295618 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:37.295661 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:37.298981 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:37.299743 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:37.299759 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:37.299767 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:37.299939 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:37.300334 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:37.300340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:37.300343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:37.300920 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:37.301254 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:37.301742 (+ 488us) server_negotiation.cc:299] Negotiation successful
0619 14:56:37.302210 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":411,"threads_started":1}
I20260619 14:56:37.305625 14476 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f390796e0440e95dab758b0254e67" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41631 } }
I20260619 14:56:37.306140 14476 leader_election.cc:302] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd3f390796e0440e95dab758b0254e67; no voters:
I20260619 14:56:37.306865 14476 leader_election.cc:288] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:37.307024 14481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:37.307708 14481 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [term 1 LEADER]: Becoming Leader. State: Replica: dd3f390796e0440e95dab758b0254e67, State: Running, Role: LEADER
I20260619 14:56:37.308187 14481 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f390796e0440e95dab758b0254e67" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41631 } }
I20260619 14:56:37.308938 14476 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:37.311158 14482 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd3f390796e0440e95dab758b0254e67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f390796e0440e95dab758b0254e67" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41631 } } }
I20260619 14:56:37.311653 14482 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:37.311123 14483 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd3f390796e0440e95dab758b0254e67. Latest consensus state: current_term: 1 leader_uuid: "dd3f390796e0440e95dab758b0254e67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3f390796e0440e95dab758b0254e67" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41631 } } }
I20260619 14:56:37.312606 14483 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:37.316845 14486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:37.322074 14486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:37.331867 14486 catalog_manager.cc:1347] Generated new cluster ID: 6d1dc23329704390b224789e62a47737
I20260619 14:56:37.332044 14486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:37.353137 14486 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:37.354800 14486 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:37.364398 14486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd3f390796e0440e95dab758b0254e67: Generated new TSK 0
I20260619 14:56:37.365636 14486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:56:37.422465 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41631
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:42013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:37.763468 14502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:37.763764 14502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:37.763861 14502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:37.771282 14502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:37.771406 14502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:37.771522 14502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:56:37.781689 14502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:42013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:37.782977 14502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:37.784237 14502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:37.797627 14510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:37.797572 14508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:37.797605 14507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:37.804030 14502 server_base.cc:1034] running on GCE node
I20260619 14:56:37.804744 14502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:37.805825 14502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:37.807047 14502 hybrid_clock.cc:648] HybridClock initialized: now 1781880997806996 us; error 43 us; skew 500 ppm
I20260619 14:56:37.807392 14502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:37 dist-test-slave-76rk krb5kdc[14396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880997, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:37.810971 14502 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:56:37.812574 14502 webserver.cc:460] Webserver started at http://127.11.175.193:41697/ using document root <none> and password file <none>
I20260619 14:56:37.813429 14502 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:37.813575 14502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:37.813899 14502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:37.817189 14502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "a95a7a2137d24845b99275ebc7c0bd07"
format_stamp: "Formatted at 2026-06-19 14:56:37 on dist-test-slave-76rk"
server_key: "c7fbd77b0911e92fc0a1414a6b730594"
server_key_iv: "1d179477a9ef2fe361e5487ed9bf3916"
server_key_version: "encryptionkey@0"
I20260619 14:56:37.818074 14502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "a95a7a2137d24845b99275ebc7c0bd07"
format_stamp: "Formatted at 2026-06-19 14:56:37 on dist-test-slave-76rk"
server_key: "c7fbd77b0911e92fc0a1414a6b730594"
server_key_iv: "1d179477a9ef2fe361e5487ed9bf3916"
server_key_version: "encryptionkey@0"
I20260619 14:56:37.823671 14502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 14:56:37.827716 14517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:37.829260 14502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 14:56:37.829474 14502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "a95a7a2137d24845b99275ebc7c0bd07"
format_stamp: "Formatted at 2026-06-19 14:56:37 on dist-test-slave-76rk"
server_key: "c7fbd77b0911e92fc0a1414a6b730594"
server_key_iv: "1d179477a9ef2fe361e5487ed9bf3916"
server_key_version: "encryptionkey@0"
I20260619 14:56:37.829708 14502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:37.851994 14502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:37.857084 14502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:37.857407 14502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:37.858471 14502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:37.859963 14502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:37.860072 14502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:37.860224 14502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:37.860298 14502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:37.895310 14502 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:40037
I20260619 14:56:37.895409 14629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:40037 every 8 connection(s)
I20260619 14:56:37.896613 14502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:56:37.906189 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 14502
I20260619 14:56:37.906435 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:56:37.907253 11967 external_mini_cluster.cc:1428] Setting key edd1fd51233bc305ea8b6b6041592fbe
Jun 19 14:56:37 dist-test-slave-76rk krb5kdc[14396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880997, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:37.919713 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:37.899358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:37267 (local address 127.11.175.254:41631)
0619 14:56:37.899830 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:37.899837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:37.900739 (+ 902us) server_negotiation.cc:407] Connection header received
0619 14:56:37.902723 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:37.902729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:37.902826 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:37.902953 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:37.904585 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:37.905334 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:37.907915 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:37.908491 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:37.908674 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:37.913385 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:37.913390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:37.913393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:37.913437 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:37.916190 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:37.917139 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:37.917148 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:37.917156 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:37.917304 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:37.917762 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:37.917767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:37.917772 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:37.918072 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:37.918234 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:37.919076 (+ 842us) server_negotiation.cc:299] Negotiation successful
0619 14:56:37.919337 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":250,"threads_started":1}
I20260619 14:56:37.920751 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:37.899785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41631 (local address 127.11.175.193:37267)
0619 14:56:37.900548 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:37.900592 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:37.902388 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:37.903211 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:37.903221 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:37.903782 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:37.904383 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:37.904400 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:37.905523 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:37.905527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:37.907718 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:37.907733 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:37.908830 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:37.908840 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:37.909137 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:37.909915 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:37.909939 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:37.913020 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:37.916364 (+ 3344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:37.916375 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:37.916396 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:37.916986 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:37.917429 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:37.917434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:37.917438 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:37.917639 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:37.918209 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:37.918217 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:37.918594 (+ 377us) client_negotiation.cc:769] Sending connection context
0619 14:56:37.918961 (+ 367us) client_negotiation.cc:240] Negotiation successful
0619 14:56:37.919552 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":313,"threads_started":1}
I20260619 14:56:37.922720 14630 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41631
I20260619 14:56:37.923080 14630 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:37.923866 14630 heartbeater.cc:511] Master 127.11.175.254:41631 requested a full tablet report, sending...
I20260619 14:56:37.926718 14441 ts_manager.cc:194] Registered new tserver with Master: a95a7a2137d24845b99275ebc7c0bd07 (127.11.175.193:40037)
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 14:56:37.929075 14441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:37267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:56:37.955904 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41631
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:42013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:38.282449 14636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:38.282819 14636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:38.282938 14636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:38.290362 14636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:38.290477 14636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:38.290613 14636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:56:38.300864 14636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:42013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:38.302102 14636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:38.303442 14636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:38.316872 14642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:38.316969 14641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:38.316948 14644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:38.317818 14636 server_base.cc:1034] running on GCE node
I20260619 14:56:38.318359 14636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:38.319375 14636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:38.320600 14636 hybrid_clock.cc:648] HybridClock initialized: now 1781880998320511 us; error 76 us; skew 500 ppm
I20260619 14:56:38.320986 14636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:38 dist-test-slave-76rk krb5kdc[14396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880998, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:38.331336 14636 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:56:38.333022 14636 webserver.cc:460] Webserver started at http://127.11.175.194:46699/ using document root <none> and password file <none>
I20260619 14:56:38.333812 14636 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:38.333966 14636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:38.334322 14636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:38.337481 14636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "09acc3ccffeb4b63aa763303e4798af4"
format_stamp: "Formatted at 2026-06-19 14:56:38 on dist-test-slave-76rk"
server_key: "4bf916d872d7bc789ae7f43f21b8befc"
server_key_iv: "3d74847219d4636aa5ad6b492383d14c"
server_key_version: "encryptionkey@0"
I20260619 14:56:38.338318 14636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "09acc3ccffeb4b63aa763303e4798af4"
format_stamp: "Formatted at 2026-06-19 14:56:38 on dist-test-slave-76rk"
server_key: "4bf916d872d7bc789ae7f43f21b8befc"
server_key_iv: "3d74847219d4636aa5ad6b492383d14c"
server_key_version: "encryptionkey@0"
I20260619 14:56:38.344107 14636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 14:56:38.348357 14651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:38.349920 14636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:38.350122 14636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "09acc3ccffeb4b63aa763303e4798af4"
format_stamp: "Formatted at 2026-06-19 14:56:38 on dist-test-slave-76rk"
server_key: "4bf916d872d7bc789ae7f43f21b8befc"
server_key_iv: "3d74847219d4636aa5ad6b492383d14c"
server_key_version: "encryptionkey@0"
I20260619 14:56:38.350355 14636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:38.366732 14636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:38.372040 14636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:38.372350 14636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:38.373338 14636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:38.374821 14636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:38.374928 14636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:38.375082 14636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:38.375152 14636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:38.409570 14636 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:39865
I20260619 14:56:38.409621 14763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:39865 every 8 connection(s)
I20260619 14:56:38.410992 14636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:56:38.416443 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 14636
I20260619 14:56:38.416658 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:56:38.417205 11967 external_mini_cluster.cc:1428] Setting key 61d33cf258fd9652b0cdde150b9294d6
Jun 19 14:56:38 dist-test-slave-76rk krb5kdc[14396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880998, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:38.431661 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:38.413341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:33565 (local address 127.11.175.254:41631)
0619 14:56:38.413542 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:38.413548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:38.414742 (+ 1194us) server_negotiation.cc:407] Connection header received
0619 14:56:38.416780 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:38.416787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:38.416890 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:38.416999 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:38.418746 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:38.419692 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:38.421207 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:38.421958 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:38.422100 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:38.425501 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:38.425507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:38.425513 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:38.425560 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:38.428375 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:38.429309 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:38.429312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:38.429315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:38.429446 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:38.429960 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:38.429963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:38.429965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:38.430215 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:38.430321 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:38.431235 (+ 914us) server_negotiation.cc:299] Negotiation successful
0619 14:56:38.431422 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 14:56:38.433231 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:38.413876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41631 (local address 127.11.175.194:33565)
0619 14:56:38.414567 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:38.414605 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:38.416406 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:38.417209 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:38.417218 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:38.417851 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:38.418553 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:38.418576 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:38.419850 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:38.419854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:38.421062 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:38.421074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:38.422168 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:38.422173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:38.422384 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:38.423102 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:38.423125 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:38.425260 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:38.428516 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:38.428529 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:38.428548 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:38.429191 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:38.429559 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:38.429567 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:38.429573 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:38.429849 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:38.430340 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:38.430351 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:38.430827 (+ 476us) client_negotiation.cc:769] Sending connection context
0619 14:56:38.431150 (+ 323us) client_negotiation.cc:240] Negotiation successful
0619 14:56:38.431711 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":266,"threads_started":1}
I20260619 14:56:38.434841 14764 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41631
I20260619 14:56:38.435202 14764 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:38.436216 14764 heartbeater.cc:511] Master 127.11.175.254:41631 requested a full tablet report, sending...
I20260619 14:56:38.437835 14441 ts_manager.cc:194] Registered new tserver with Master: 09acc3ccffeb4b63aa763303e4798af4 (127.11.175.194:39865)
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
I20260619 14:56:38.438767 14441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:33565
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:56:38.466207 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41631
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:42013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:38.794456 14769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:38.794800 14769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:38.794963 14769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:38.802278 14769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:38.802398 14769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:38.802572 14769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:56:38.813005 14769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:42013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:38.814455 14769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:38.815883 14769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:38.830469 14774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:38.830493 14775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:38.830708 14777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:38.831702 14769 server_base.cc:1034] running on GCE node
I20260619 14:56:38.832397 14769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:38.833441 14769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:38.834688 14769 hybrid_clock.cc:648] HybridClock initialized: now 1781880998834575 us; error 60 us; skew 500 ppm
I20260619 14:56:38.835058 14769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:38 dist-test-slave-76rk krb5kdc[14396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880998, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:38.838332 14769 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:56:38.839818 14769 webserver.cc:460] Webserver started at http://127.11.175.195:45807/ using document root <none> and password file <none>
I20260619 14:56:38.840502 14769 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:38.840602 14769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:38.840865 14769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:38.843853 14769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "68d95c00215c4cd3af13282a49b1a77a"
format_stamp: "Formatted at 2026-06-19 14:56:38 on dist-test-slave-76rk"
server_key: "496e56e69802d75ac285c3bb8d8faa8d"
server_key_iv: "edc0fc3bf6fb9a2615bd988fb4821a92"
server_key_version: "encryptionkey@0"
I20260619 14:56:38.844643 14769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "68d95c00215c4cd3af13282a49b1a77a"
format_stamp: "Formatted at 2026-06-19 14:56:38 on dist-test-slave-76rk"
server_key: "496e56e69802d75ac285c3bb8d8faa8d"
server_key_iv: "edc0fc3bf6fb9a2615bd988fb4821a92"
server_key_version: "encryptionkey@0"
I20260619 14:56:38.850010 14769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 14:56:38.853812 14784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:38.855543 14769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:56:38.855741 14769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "68d95c00215c4cd3af13282a49b1a77a"
format_stamp: "Formatted at 2026-06-19 14:56:38 on dist-test-slave-76rk"
server_key: "496e56e69802d75ac285c3bb8d8faa8d"
server_key_iv: "edc0fc3bf6fb9a2615bd988fb4821a92"
server_key_version: "encryptionkey@0"
I20260619 14:56:38.855965 14769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:38.876494 14769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:38.881399 14769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:38.881728 14769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:38.882975 14769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:38.884460 14769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:38.884568 14769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:38.884764 14769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:38.884873 14769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:38.920426 14769 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:39265
I20260619 14:56:38.920516 14896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:39265 every 8 connection(s)
I20260619 14:56:38.921762 14769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:56:38.927093 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 14769
I20260619 14:56:38.927289 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:56:38.927723 11967 external_mini_cluster.cc:1428] Setting key 63447cccb228fd70e8afe991a7a580a7
I20260619 14:56:38.932865 14630 heartbeater.cc:503] Master 127.11.175.254:41631 was elected leader, sending a full tablet report...
Jun 19 14:56:38 dist-test-slave-76rk krb5kdc[14396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781880998, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:38.944648 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:38.924567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:39813 (local address 127.11.175.254:41631)
0619 14:56:38.924798 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:38.924804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:38.926025 (+ 1221us) server_negotiation.cc:407] Connection header received
0619 14:56:38.928157 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:38.928166 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:38.928282 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:38.928395 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:38.930848 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:38.931791 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:38.933844 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:38.934451 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:38.934595 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:38.938396 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:38.938401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:38.938405 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:38.938439 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:38.941025 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:38.941820 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:38.941823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:38.941826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:38.942050 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:38.942734 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:38.942739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:38.942743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:38.943136 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:38.943322 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:38.944212 (+ 890us) server_negotiation.cc:299] Negotiation successful
0619 14:56:38.944403 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 14:56:38.945619 14899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:38.925038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41631 (local address 127.11.175.195:39813)
0619 14:56:38.925854 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:38.925893 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:38.927864 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:38.928924 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:38.928937 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:38.929719 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:38.930598 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:38.930627 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:38.931961 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:38.931966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:38.933676 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:38.933694 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:38.934776 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:38.934783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:38.935001 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:38.935762 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:38.935792 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:38.938149 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:38.941174 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:38.941181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:38.941195 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:38.941679 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:38.942201 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:38.942207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:38.942210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:38.942538 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:38.943276 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:38.943282 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:38.943804 (+ 522us) client_negotiation.cc:769] Sending connection context
0619 14:56:38.944099 (+ 295us) client_negotiation.cc:240] Negotiation successful
0619 14:56:38.944507 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":330,"threads_started":1}
I20260619 14:56:38.947213 14897 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41631
I20260619 14:56:38.947542 14897 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:38.948293 14897 heartbeater.cc:511] Master 127.11.175.254:41631 requested a full tablet report, sending...
I20260619 14:56:38.949847 14441 ts_manager.cc:194] Registered new tserver with Master: 68d95c00215c4cd3af13282a49b1a77a (127.11.175.195:39265)
I20260619 14:56:38.950910 14441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:39813
I20260619 14:56:38.956452 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 14:56:38.989682 14632 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:56:38.986962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.11.175.254:41631)
0619 14:56:38.987308 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:38.987317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:38.987689 (+ 372us) server_negotiation.cc:407] Connection header received
0619 14:56:38.987957 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:38.987962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:38.988028 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:38.988137 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:38.989461 (+ 1324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46654: BlockingRecv error: recv got EOF from 127.0.0.1:46654 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 14:56:38.992739 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 14502
W20260619 14:56:38.995966 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 14502: Not authorized: Client connection negotiation failed: client connection to 127.11.175.193:40037: server requires authentication, but client does not have Kerberos credentials available
I20260619 14:56:38.996160 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 14502
I20260619 14:56:39.012442 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 14636
W20260619 14:56:39.015234 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 14636: Not authorized: Client connection negotiation failed: client connection to 127.11.175.194:39865: server requires authentication, but client does not have Kerberos credentials available
I20260619 14:56:39.015434 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 14636
I20260619 14:56:39.031425 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 14769
W20260619 14:56:39.034427 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 14769: Not authorized: Client connection negotiation failed: client connection to 127.11.175.195:39265: server requires authentication, but client does not have Kerberos credentials available
I20260619 14:56:39.034565 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 14769
I20260619 14:56:39.050289 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 14409
I20260619 14:56:39.051600 14471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:39.158185 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 14409
2026-06-19T14:56:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3661 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:56:39 dist-test-slave-76rk krb5kdc[14914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:56:39 dist-test-slave-76rk krb5kdc[14914](info): set up 2 sockets
Jun 19 14:56:39 dist-test-slave-76rk krb5kdc[14914](info): commencing operation
krb5kdc: starting...
W20260619 14:56:40.255136 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.001s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:56:40 dist-test-slave-76rk krb5kdc[14914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:56:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:56:40.480523 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:37573
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:40497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:37573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:40.816608 14927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:40.817040 14927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:40.817229 14927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:40.825011 14927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:40.825212 14927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:40.825340 14927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:40.825457 14927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:40.825562 14927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:56:40.836207 14927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:37573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:37573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:40.837689 14927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:40.839181 14927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:40.852671 14933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:40.852598 14935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:40.853394 14927 server_base.cc:1034] running on GCE node
W20260619 14:56:40.853557 14932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:40.854835 14927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:40.856359 14927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:40.857653 14927 hybrid_clock.cc:648] HybridClock initialized: now 1781881000857602 us; error 46 us; skew 500 ppm
I20260619 14:56:40.858127 14927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:40 dist-test-slave-76rk krb5kdc[14914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881000, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:40.861804 14927 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:56:40.865001 14927 webserver.cc:460] Webserver started at http://127.11.175.254:40877/ using document root <none> and password file <none>
I20260619 14:56:40.866312 14927 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:40.866484 14927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:40.866925 14927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:40.872913 14927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "24b1cfa7be1d43b79122928b1048e4bc"
format_stamp: "Formatted at 2026-06-19 14:56:40 on dist-test-slave-76rk"
server_key: "891ea2cd7868544fbcbebf8010dabae8"
server_key_iv: "b474f05f68974db08853e4ac840daf34"
server_key_version: "encryptionkey@0"
I20260619 14:56:40.874444 14927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "24b1cfa7be1d43b79122928b1048e4bc"
format_stamp: "Formatted at 2026-06-19 14:56:40 on dist-test-slave-76rk"
server_key: "891ea2cd7868544fbcbebf8010dabae8"
server_key_iv: "b474f05f68974db08853e4ac840daf34"
server_key_version: "encryptionkey@0"
I20260619 14:56:40.883682 14927 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260619 14:56:40.888621 14942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:40.890645 14927 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260619 14:56:40.890920 14927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "24b1cfa7be1d43b79122928b1048e4bc"
format_stamp: "Formatted at 2026-06-19 14:56:40 on dist-test-slave-76rk"
server_key: "891ea2cd7868544fbcbebf8010dabae8"
server_key_iv: "b474f05f68974db08853e4ac840daf34"
server_key_version: "encryptionkey@0"
I20260619 14:56:40.891180 14927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:40.907709 14927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:40.912294 14927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:40.912596 14927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:40.932358 14927 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:37573
I20260619 14:56:40.932394 14993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:37573 every 8 connection(s)
I20260619 14:56:40.933847 14927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:40.937754 14994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:40.941366 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 14927
I20260619 14:56:40.941612 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:40.942296 11967 external_mini_cluster.cc:1428] Setting key a33488e752427e65969495aa3af090c2
I20260619 14:56:40.951388 14994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc: Bootstrap starting.
I20260619 14:56:40.955163 14994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:40.956210 14994 log.cc:824] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc: Log is configured to *not* fsync() on all Append() calls
Jun 19 14:56:40 dist-test-slave-76rk krb5kdc[14914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:40.959277 14994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc: No bootstrap required, opened a new log
I20260619 14:56:40.967298 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:40.944281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.11.175.254:37573)
0619 14:56:40.945146 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:40.945163 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:40.945212 (+ 49us) server_negotiation.cc:407] Connection header received
0619 14:56:40.947061 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:40.947098 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:40.947692 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:40.948169 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:40.949611 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:40.951609 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:40.952959 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:40.954022 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:40.954207 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:40.958512 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:40.958523 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:40.958544 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:40.958590 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:40.962781 (+ 4191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:40.963471 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:40.963480 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:40.963488 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:40.963695 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:40.964065 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:40.964071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:40.964074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:40.964871 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:40.965166 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:40.965550 (+ 384us) server_negotiation.cc:299] Negotiation successful
0619 14:56:40.966012 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":366,"threads_started":1}
I20260619 14:56:40.970216 14994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b1cfa7be1d43b79122928b1048e4bc" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37573 } }
I20260619 14:56:40.970744 14994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:40.970893 14994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24b1cfa7be1d43b79122928b1048e4bc, State: Initialized, Role: FOLLOWER
I20260619 14:56:40.971517 14994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b1cfa7be1d43b79122928b1048e4bc" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37573 } }
I20260619 14:56:40.971818 14994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:40.971910 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 14:56:40.971979 14994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:40.972173 14994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:40.974607 14994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b1cfa7be1d43b79122928b1048e4bc" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37573 } }
I20260619 14:56:40.975270 14994 leader_election.cc:302] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24b1cfa7be1d43b79122928b1048e4bc; no voters:
I20260619 14:56:40.975916 14994 leader_election.cc:288] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:40.976079 14999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:40.978048 14994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:40.983198 14999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [term 1 LEADER]: Becoming Leader. State: Replica: 24b1cfa7be1d43b79122928b1048e4bc, State: Running, Role: LEADER
I20260619 14:56:40.983803 14999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b1cfa7be1d43b79122928b1048e4bc" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37573 } }
I20260619 14:56:40.987143 15000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24b1cfa7be1d43b79122928b1048e4bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b1cfa7be1d43b79122928b1048e4bc" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37573 } } }
I20260619 14:56:40.987574 15000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:40.992480 15001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24b1cfa7be1d43b79122928b1048e4bc. Latest consensus state: current_term: 1 leader_uuid: "24b1cfa7be1d43b79122928b1048e4bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b1cfa7be1d43b79122928b1048e4bc" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37573 } } }
I20260619 14:56:40.992866 15001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc [sys.catalog]: This master's current role is: LEADER
W20260619 14:56:41.005012 15014 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 14:56:41.005190 15014 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 14:56:41.005313 15015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:41.007026 15015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:41.017009 15015 catalog_manager.cc:1347] Generated new cluster ID: f0cfab5de2fa4560bbf881beab451ca1
I20260619 14:56:41.017133 15015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:41.029045 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:41.019685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60406 (local address 127.11.175.254:37573)
0619 14:56:41.019865 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:41.019871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:41.020460 (+ 589us) server_negotiation.cc:407] Connection header received
0619 14:56:41.020653 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:41.020659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:41.020737 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:41.020847 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:41.022138 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:41.023274 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:41.025178 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:41.025855 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:41.025972 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:41.027327 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:41.027334 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:41.027339 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:41.028193 (+ 854us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:41.028272 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:41.028318 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:41.028423 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:41.028605 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 14:56:41.028713 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 14:56:41.041167 15015 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:41.042539 15015 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:41.050757 15015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24b1cfa7be1d43b79122928b1048e4bc: Generated new TSK 0
I20260619 14:56:41.051961 15015 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 14:56:41.067615 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 14927
I20260619 14:56:41.068881 14989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:41.161593 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 14927
2026-06-19T14:56:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1997 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:56:41 dist-test-slave-76rk krb5kdc[15028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:56:41 dist-test-slave-76rk krb5kdc[15028](info): set up 2 sockets
Jun 19 14:56:41 dist-test-slave-76rk krb5kdc[15028](info): commencing operation
krb5kdc: starting...
W20260619 14:56:42.255129 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:56:42 dist-test-slave-76rk krb5kdc[15028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:56:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:56:42.466239 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:40001
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:40001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:42.797192 15041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:42.797561 15041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:42.797726 15041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:42.804921 15041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:42.805075 15041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:42.805193 15041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:42.805300 15041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:42.805389 15041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:56:42.815608 15041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:40001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:40001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:42.817026 15041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:42.818379 15041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:42.830487 15047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:42.830523 15046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:42.830412 15049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:42.831641 15041 server_base.cc:1034] running on GCE node
I20260619 14:56:42.832501 15041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:42.833977 15041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:42.835280 15041 hybrid_clock.cc:648] HybridClock initialized: now 1781881002835229 us; error 46 us; skew 500 ppm
I20260619 14:56:42.835711 15041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:42 dist-test-slave-76rk krb5kdc[15028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881002, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:42.839329 15041 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:56:42.840878 15041 webserver.cc:460] Webserver started at http://127.11.175.254:39385/ using document root <none> and password file <none>
I20260619 14:56:42.841581 15041 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:42.841689 15041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:42.841946 15041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:42.845314 15041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "cdc5249eaf154f399fb0a405269d8e09"
format_stamp: "Formatted at 2026-06-19 14:56:42 on dist-test-slave-76rk"
server_key: "842ba3a20e203bfef195d3db10011e7b"
server_key_iv: "15fa6af6828ec95b9a49499be05d77dc"
server_key_version: "encryptionkey@0"
I20260619 14:56:42.846076 15041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "cdc5249eaf154f399fb0a405269d8e09"
format_stamp: "Formatted at 2026-06-19 14:56:42 on dist-test-slave-76rk"
server_key: "842ba3a20e203bfef195d3db10011e7b"
server_key_iv: "15fa6af6828ec95b9a49499be05d77dc"
server_key_version: "encryptionkey@0"
I20260619 14:56:42.851570 15041 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 14:56:42.855744 15056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:42.857309 15041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 14:56:42.857462 15041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "cdc5249eaf154f399fb0a405269d8e09"
format_stamp: "Formatted at 2026-06-19 14:56:42 on dist-test-slave-76rk"
server_key: "842ba3a20e203bfef195d3db10011e7b"
server_key_iv: "15fa6af6828ec95b9a49499be05d77dc"
server_key_version: "encryptionkey@0"
I20260619 14:56:42.857641 15041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:42.877741 15041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:42.882462 15041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:42.882828 15041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:42.902423 15041 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:40001
I20260619 14:56:42.902470 15107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:40001 every 8 connection(s)
I20260619 14:56:42.904026 15041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:42.906960 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 15041
I20260619 14:56:42.907199 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:42.908004 11967 external_mini_cluster.cc:1428] Setting key ae018988240a11d4dbbff9f13a2b3451
I20260619 14:56:42.908901 15108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:42.924885 15108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09: Bootstrap starting.
Jun 19 14:56:42 dist-test-slave-76rk krb5kdc[15028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:42.929306 15108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:42.930855 15108 log.cc:824] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:42.934958 15108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09: No bootstrap required, opened a new log
I20260619 14:56:42.936197 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:42.910272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.11.175.254:40001)
0619 14:56:42.911190 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:42.911204 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:42.911248 (+ 44us) server_negotiation.cc:407] Connection header received
0619 14:56:42.913879 (+ 2631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:42.913909 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:42.914421 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:42.914928 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:42.916826 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:42.918232 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:42.920118 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:42.920949 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:42.921108 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:42.926230 (+ 5122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:42.926251 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:42.926274 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:42.926336 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:42.930830 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:42.931992 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:42.932001 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:42.932008 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:42.932194 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:42.932721 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:42.932727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:42.932730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:42.933456 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:42.933778 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:42.934199 (+ 421us) server_negotiation.cc:299] Negotiation successful
0619 14:56:42.934732 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":488,"threads_started":1}
I20260619 14:56:42.944226 15108 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5249eaf154f399fb0a405269d8e09" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 40001 } }
I20260619 14:56:42.944619 15108 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:42.944691 15108 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdc5249eaf154f399fb0a405269d8e09, State: Initialized, Role: FOLLOWER
I20260619 14:56:42.945216 15108 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5249eaf154f399fb0a405269d8e09" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 40001 } }
I20260619 14:56:42.945433 15108 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:42.945534 15108 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:42.945658 15108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:42.948194 15108 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5249eaf154f399fb0a405269d8e09" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 40001 } }
I20260619 14:56:42.948665 15108 leader_election.cc:302] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdc5249eaf154f399fb0a405269d8e09; no voters:
I20260619 14:56:42.949241 15108 leader_election.cc:288] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:42.949426 15113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:42.950421 15113 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [term 1 LEADER]: Becoming Leader. State: Replica: cdc5249eaf154f399fb0a405269d8e09, State: Running, Role: LEADER
I20260619 14:56:42.951007 15113 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5249eaf154f399fb0a405269d8e09" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 40001 } }
I20260619 14:56:42.951772 15108 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:42.953858 15115 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdc5249eaf154f399fb0a405269d8e09. Latest consensus state: current_term: 1 leader_uuid: "cdc5249eaf154f399fb0a405269d8e09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5249eaf154f399fb0a405269d8e09" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 40001 } } }
I20260619 14:56:42.954272 15115 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:42.954864 15114 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdc5249eaf154f399fb0a405269d8e09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5249eaf154f399fb0a405269d8e09" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 40001 } } }
I20260619 14:56:42.955286 15114 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:42.959528 15122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:42.964432 15122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:42.973757 15122 catalog_manager.cc:1347] Generated new cluster ID: 8521c0fe18d14559b12a3b7ecf0b9e83
I20260619 14:56:42.973892 15122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:42.999092 15122 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:43.001184 15122 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:43.017546 15122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdc5249eaf154f399fb0a405269d8e09: Generated new TSK 0
I20260619 14:56:43.018461 15122 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:56:43.068524 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:40001
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:43.407908 15134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:43.408183 15134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:43.408277 15134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:43.415498 15134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:43.415619 15134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:43.415735 15134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:56:43.425776 15134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:40001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:43.427103 15134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:43.428341 15134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:43.441174 15139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:43.441270 15142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:43.441264 15140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:43.442449 15134 server_base.cc:1034] running on GCE node
I20260619 14:56:43.443161 15134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:43.444132 15134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:43.445338 15134 hybrid_clock.cc:648] HybridClock initialized: now 1781881003445272 us; error 58 us; skew 500 ppm
I20260619 14:56:43.445626 15134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:43 dist-test-slave-76rk krb5kdc[15028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881003, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:43.454283 15134 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:56:43.455972 15134 webserver.cc:460] Webserver started at http://127.11.175.193:42661/ using document root <none> and password file <none>
I20260619 14:56:43.456692 15134 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:43.456800 15134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:43.457156 15134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:43.460390 15134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "97102e46b4eb433db207fe6463450e6e"
format_stamp: "Formatted at 2026-06-19 14:56:43 on dist-test-slave-76rk"
server_key: "af0b7505df5457d009ef3a23171fd9d7"
server_key_iv: "ae62293a2eab9338d4e3722febaa2b6b"
server_key_version: "encryptionkey@0"
I20260619 14:56:43.461225 15134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "97102e46b4eb433db207fe6463450e6e"
format_stamp: "Formatted at 2026-06-19 14:56:43 on dist-test-slave-76rk"
server_key: "af0b7505df5457d009ef3a23171fd9d7"
server_key_iv: "ae62293a2eab9338d4e3722febaa2b6b"
server_key_version: "encryptionkey@0"
I20260619 14:56:43.466459 15134 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 14:56:43.470001 15149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:43.471490 15134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:56:43.471678 15134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "97102e46b4eb433db207fe6463450e6e"
format_stamp: "Formatted at 2026-06-19 14:56:43 on dist-test-slave-76rk"
server_key: "af0b7505df5457d009ef3a23171fd9d7"
server_key_iv: "ae62293a2eab9338d4e3722febaa2b6b"
server_key_version: "encryptionkey@0"
I20260619 14:56:43.471909 15134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:43.484362 15134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:43.488726 15134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:43.489042 15134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:43.489981 15134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:43.491350 15134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:43.491453 15134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:43.491602 15134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:43.491681 15134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:43.526186 15134 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:46485
I20260619 14:56:43.526227 15261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:46485 every 8 connection(s)
I20260619 14:56:43.527645 15134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:56:43.528910 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 15134
I20260619 14:56:43.529136 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:56:43.529997 11967 external_mini_cluster.cc:1428] Setting key 85215f2ff57e7dfa23c510093d35f3fd
Jun 19 14:56:43 dist-test-slave-76rk krb5kdc[15028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881003, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 14:56:43.551731 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:43.531024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:56617 (local address 127.11.175.254:40001)
0619 14:56:43.531500 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:43.531505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:43.532686 (+ 1181us) server_negotiation.cc:407] Connection header received
0619 14:56:43.534532 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:43.534541 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:43.534662 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:43.534782 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:43.536364 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:43.537492 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:43.539985 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:43.540378 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:43.540489 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:43.545037 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:43.545043 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:43.545046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:43.545083 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:43.547958 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:43.549091 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:43.549095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:43.549098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:43.549227 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:43.549754 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:43.549757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:43.549759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:43.550050 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:43.550165 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:43.551203 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0619 14:56:43.551381 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":235,"threads_started":1}
I20260619 14:56:43.553018 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:43.531670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:40001 (local address 127.11.175.193:56617)
0619 14:56:43.532538 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:43.532587 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:43.534247 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:43.534993 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:43.535009 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:43.535575 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:43.536179 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:43.536196 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:43.537648 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:43.537652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:43.539831 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:43.539849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:43.540612 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:43.540621 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:43.540953 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:43.541692 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:43.541725 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:43.544786 (+ 3061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:43.548149 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:43.548162 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:43.548185 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:43.548942 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:43.549348 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:43.549353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:43.549359 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:43.549639 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:43.550189 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:43.550197 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:43.550752 (+ 555us) client_negotiation.cc:769] Sending connection context
0619 14:56:43.551103 (+ 351us) client_negotiation.cc:240] Negotiation successful
0619 14:56:43.551730 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":369,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
I20260619 14:56:43.554741 15262 heartbeater.cc:347] Connected to a master server at 127.11.175.254:40001
I20260619 14:56:43.555114 15262 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:43.555883 15262 heartbeater.cc:511] Master 127.11.175.254:40001 requested a full tablet report, sending...
I20260619 14:56:43.557993 15073 ts_manager.cc:194] Registered new tserver with Master: 97102e46b4eb433db207fe6463450e6e (127.11.175.193:46485)
I20260619 14:56:43.559561 15073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:56617
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:56:43.583397 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:40001
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:43.910763 15268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:43.911140 15268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:43.911336 15268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:43.918881 15268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:43.919037 15268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:43.919229 15268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:56:43.929075 15268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:40001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:43.930425 15268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:43.931736 15268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:43.945464 15273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:43.945501 15274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:43.945425 15276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:43.946367 15268 server_base.cc:1034] running on GCE node
I20260619 14:56:43.947094 15268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:43.948171 15268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:43.949416 15268 hybrid_clock.cc:648] HybridClock initialized: now 1781881003949369 us; error 52 us; skew 500 ppm
I20260619 14:56:43.949764 15268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:43 dist-test-slave-76rk krb5kdc[15028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881003, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:43.953289 15268 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:56:43.954982 15268 webserver.cc:460] Webserver started at http://127.11.175.194:45099/ using document root <none> and password file <none>
I20260619 14:56:43.955730 15268 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:43.955834 15268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:43.956193 15268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:43.959270 15268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "ffc09d1883074a758bb5aa28fe2f0cf4"
format_stamp: "Formatted at 2026-06-19 14:56:43 on dist-test-slave-76rk"
server_key: "d496067ef53f64bc588fcfd4e534526d"
server_key_iv: "6ee6f4e631adc6977bd5540a0c961b50"
server_key_version: "encryptionkey@0"
I20260619 14:56:43.960094 15268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "ffc09d1883074a758bb5aa28fe2f0cf4"
format_stamp: "Formatted at 2026-06-19 14:56:43 on dist-test-slave-76rk"
server_key: "d496067ef53f64bc588fcfd4e534526d"
server_key_iv: "6ee6f4e631adc6977bd5540a0c961b50"
server_key_version: "encryptionkey@0"
I20260619 14:56:43.966496 15268 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260619 14:56:43.970515 15283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:43.972154 15268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:56:43.972380 15268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "ffc09d1883074a758bb5aa28fe2f0cf4"
format_stamp: "Formatted at 2026-06-19 14:56:43 on dist-test-slave-76rk"
server_key: "d496067ef53f64bc588fcfd4e534526d"
server_key_iv: "6ee6f4e631adc6977bd5540a0c961b50"
server_key_version: "encryptionkey@0"
I20260619 14:56:43.972574 15268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:43.995194 15268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:43.999979 15268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:44.000240 15268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:44.001188 15268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:44.002538 15268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:44.002619 15268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:44.002832 15268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:44.002916 15268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:44.036806 15268 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:39447
I20260619 14:56:44.036850 15395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:39447 every 8 connection(s)
I20260619 14:56:44.038120 15268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:56:44.043830 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 15268
I20260619 14:56:44.044037 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:56:44.044495 11967 external_mini_cluster.cc:1428] Setting key febc2c54df154e9672a5e5fecf1e7847
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881003, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:44.061385 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:44.040898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:60683 (local address 127.11.175.254:40001)
0619 14:56:44.041096 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:44.041101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:44.042437 (+ 1336us) server_negotiation.cc:407] Connection header received
0619 14:56:44.044458 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:44.044464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:44.044558 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:44.044676 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:44.046600 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.047348 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.049935 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.050352 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.050446 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:44.055023 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:44.055028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:44.055031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:44.055064 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:44.058164 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.059034 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.059045 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.059048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.059171 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.059609 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.059613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.059615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.059901 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:44.060083 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:44.060774 (+ 691us) server_negotiation.cc:299] Negotiation successful
0619 14:56:44.061014 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 14:56:44.062476 15398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:44.041501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:40001 (local address 127.11.175.194:60683)
0619 14:56:44.042226 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:44.042263 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:44.044173 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:44.044980 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:44.044992 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:44.045603 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:44.046420 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:44.046441 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.047516 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.047525 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:44.049763 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:44.049783 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.050587 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.050597 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:44.050962 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:44.051779 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:44.051815 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:44.054785 (+ 2970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:44.058342 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:44.058354 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:44.058382 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:44.058903 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:44.059281 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:44.059286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:44.059290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:44.059480 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:44.060029 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:44.060037 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:44.060397 (+ 360us) client_negotiation.cc:769] Sending connection context
0619 14:56:44.060681 (+ 284us) client_negotiation.cc:240] Negotiation successful
0619 14:56:44.061229 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":289,"threads_started":1}
I20260619 14:56:44.063954 15396 heartbeater.cc:347] Connected to a master server at 127.11.175.254:40001
I20260619 14:56:44.064267 15396 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:44.064996 15396 heartbeater.cc:511] Master 127.11.175.254:40001 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
I20260619 14:56:44.066442 15073 ts_manager.cc:194] Registered new tserver with Master: ffc09d1883074a758bb5aa28fe2f0cf4 (127.11.175.194:39447)
I20260619 14:56:44.067389 15073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:60683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:56:44.092258 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:40001
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:44.424302 15401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:44.424590 15401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:44.424683 15401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:44.432138 15401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:44.432253 15401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:44.432367 15401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:56:44.443076 15401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:40001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:44.444334 15401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:44.445565 15401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:44.459194 15407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:44.459276 15406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:44.459285 15409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:44.460269 15401 server_base.cc:1034] running on GCE node
I20260619 14:56:44.461181 15401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:44.462380 15401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:44.463642 15401 hybrid_clock.cc:648] HybridClock initialized: now 1781881004463609 us; error 50 us; skew 500 ppm
I20260619 14:56:44.464006 15401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881004, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:44.467389 15401 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:56:44.468878 15401 webserver.cc:460] Webserver started at http://127.11.175.195:44899/ using document root <none> and password file <none>
I20260619 14:56:44.469609 15401 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:44.469758 15401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:44.470078 15401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:44.473215 15401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "e0f80a96cfe145e1866f3fcaa51d0a64"
format_stamp: "Formatted at 2026-06-19 14:56:44 on dist-test-slave-76rk"
server_key: "bf2624d89eb747c8806da75c0390d779"
server_key_iv: "f353b480c10bb485b17f9e9511294f84"
server_key_version: "encryptionkey@0"
I20260619 14:56:44.474080 15401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "e0f80a96cfe145e1866f3fcaa51d0a64"
format_stamp: "Formatted at 2026-06-19 14:56:44 on dist-test-slave-76rk"
server_key: "bf2624d89eb747c8806da75c0390d779"
server_key_iv: "f353b480c10bb485b17f9e9511294f84"
server_key_version: "encryptionkey@0"
I20260619 14:56:44.479815 15401 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260619 14:56:44.483537 15416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:44.485062 15401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:44.485198 15401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "e0f80a96cfe145e1866f3fcaa51d0a64"
format_stamp: "Formatted at 2026-06-19 14:56:44 on dist-test-slave-76rk"
server_key: "bf2624d89eb747c8806da75c0390d779"
server_key_iv: "f353b480c10bb485b17f9e9511294f84"
server_key_version: "encryptionkey@0"
I20260619 14:56:44.485383 15401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:44.516288 15401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:44.520881 15401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:44.521160 15401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:44.522248 15401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:44.523766 15401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:44.523845 15401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:44.523944 15401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:44.524003 15401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:44.561739 15401 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:43727
I20260619 14:56:44.561812 15528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:43727 every 8 connection(s)
I20260619 14:56:44.563356 15401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:56:44.563488 15262 heartbeater.cc:503] Master 127.11.175.254:40001 was elected leader, sending a full tablet report...
I20260619 14:56:44.573372 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 15401
I20260619 14:56:44.573594 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:56:44.574061 11967 external_mini_cluster.cc:1428] Setting key 950c0ef2b49d6de2aa478d7629bafd53
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881004, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:44.585654 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:44.566506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:39199 (local address 127.11.175.254:40001)
0619 14:56:44.566974 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:44.566978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:44.568071 (+ 1093us) server_negotiation.cc:407] Connection header received
0619 14:56:44.570119 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:44.570125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:44.570246 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:44.570379 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:44.572055 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.572839 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.574989 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.575627 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.575773 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:44.579949 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:44.579954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:44.579958 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:44.579994 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:44.582541 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.583439 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.583447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.583454 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.583612 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.584102 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.584105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.584107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.584382 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:44.584510 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:44.585199 (+ 689us) server_negotiation.cc:299] Negotiation successful
0619 14:56:44.585383 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":263,"threads_started":1}
I20260619 14:56:44.586611 15532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:44.567171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:40001 (local address 127.11.175.195:39199)
0619 14:56:44.567910 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:44.567948 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:44.569810 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:44.570697 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:44.570712 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:44.571260 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:44.571871 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:44.571889 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.573030 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.573034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:44.574782 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:44.574799 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.575941 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.575949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:44.576279 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:44.577164 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:44.577195 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:44.579695 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:44.582748 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:44.582755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:44.582769 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:44.583287 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:44.583749 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:44.583752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:44.583755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:44.583990 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:44.584500 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:44.584506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:44.584835 (+ 329us) client_negotiation.cc:769] Sending connection context
0619 14:56:44.585099 (+ 264us) client_negotiation.cc:240] Negotiation successful
0619 14:56:44.585492 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":289,"threads_started":1}
I20260619 14:56:44.588176 15529 heartbeater.cc:347] Connected to a master server at 127.11.175.254:40001
I20260619 14:56:44.588558 15529 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:44.589622 15529 heartbeater.cc:511] Master 127.11.175.254:40001 requested a full tablet report, sending...
I20260619 14:56:44.591192 15073 ts_manager.cc:194] Registered new tserver with Master: e0f80a96cfe145e1866f3fcaa51d0a64 (127.11.175.195:43727)
I20260619 14:56:44.592010 15073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:39199
I20260619 14:56:44.602909 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881004, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881004, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:44.654707 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:44.639590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.11.175.254:40001)
0619 14:56:44.639912 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:44.639922 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:44.640247 (+ 325us) server_negotiation.cc:407] Connection header received
0619 14:56:44.640484 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:44.640488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:44.640559 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:44.640669 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:44.642056 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.642961 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.644287 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.644741 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.644885 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:44.649139 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:44.649145 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:44.649149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:44.649191 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:44.651898 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.652611 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.652622 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.652624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.652765 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.653147 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.653152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.653155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.653496 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:44.653761 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:44.654047 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 14:56:44.654302 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
W20260619 14:56:44.655385 15073 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36894
I20260619 14:56:44.657217 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15134
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881004, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:56:44.674002 15541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:44.658456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56974 (local address 127.11.175.193:46485)
0619 14:56:44.659199 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:44.659206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:44.659231 (+ 25us) server_negotiation.cc:407] Connection header received
0619 14:56:44.659284 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:44.659290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:44.659499 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:44.659629 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:44.660727 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.661983 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.663203 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.663758 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.663867 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:44.668149 (+ 4282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:44.668160 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:44.668175 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:44.668226 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:44.671149 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.671892 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.671900 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.671902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.672052 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.672488 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.672493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.672495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.672915 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:44.673154 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:44.673441 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 14:56:44.673649 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":387,"threads_started":1}
W20260619 14:56:44.675482 15257 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56974
W20260619 14:56:44.676679 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15134: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 14:56:44.676880 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 15134
I20260619 14:56:44.694355 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15268
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881004, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:56:44.711066 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:44.695323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45706 (local address 127.11.175.194:39447)
0619 14:56:44.695957 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:44.695964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:44.695989 (+ 25us) server_negotiation.cc:407] Connection header received
0619 14:56:44.696049 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:44.696055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:44.696263 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:44.696395 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:44.697798 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.698910 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.700111 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.700656 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.700754 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:44.704984 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:44.704997 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:44.705013 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:44.705053 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:44.708116 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.708846 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.708854 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.708856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.709011 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.709398 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.709402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.709404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.709768 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:44.710042 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:44.710340 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 14:56:44.710615 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":339,"threads_started":1}
W20260619 14:56:44.712347 15391 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45706
W20260619 14:56:44.713495 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15268: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 14:56:44.713632 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 15268
I20260619 14:56:44.729581 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15401
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881004, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:56:44.748337 15543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:44.730841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.11.175.195:43727)
0619 14:56:44.731605 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:44.731612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:44.731651 (+ 39us) server_negotiation.cc:407] Connection header received
0619 14:56:44.731725 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:44.731732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:44.731948 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:44.732164 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:44.733665 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.734913 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.736281 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:44.736928 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:44.737093 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:44.742044 (+ 4951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:44.742057 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:44.742071 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:44.742110 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:44.745250 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.746042 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.746050 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.746052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.746181 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:44.746564 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:44.746570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:44.746574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:44.747010 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:44.747351 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:44.747656 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 14:56:44.747970 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":378,"threads_started":1}
W20260619 14:56:44.749930 15524 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43170
W20260619 14:56:44.751271 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15401: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 14:56:44.751497 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 15401
I20260619 14:56:44.767900 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15041
I20260619 14:56:44.769173 15103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:44.881646 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 15041
2026-06-19T14:56:44Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3726 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15547](info): set up 2 sockets
Jun 19 14:56:44 dist-test-slave-76rk krb5kdc[15547](info): commencing operation
krb5kdc: starting...
W20260619 14:56:45.995260 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:56:46 dist-test-slave-76rk krb5kdc[15547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:56:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:56:46.206446 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:35165
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:35165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:46.554436 15560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:46.554836 15560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:46.555025 15560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:46.562516 15560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:46.562686 15560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:46.562767 15560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:46.562861 15560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:46.562943 15560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:56:46.573308 15560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:35165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:35165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:46.574810 15560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:46.576222 15560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:46.588521 15565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:46.588521 15568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:46.588452 15566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:46.589427 15560 server_base.cc:1034] running on GCE node
I20260619 14:56:46.590335 15560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:46.591962 15560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:46.593214 15560 hybrid_clock.cc:648] HybridClock initialized: now 1781881006593149 us; error 56 us; skew 500 ppm
I20260619 14:56:46.593528 15560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:46 dist-test-slave-76rk krb5kdc[15547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881006, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:46.596805 15560 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:56:46.598327 15560 webserver.cc:460] Webserver started at http://127.11.175.254:45441/ using document root <none> and password file <none>
I20260619 14:56:46.599084 15560 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:46.599184 15560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:46.599443 15560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:46.602447 15560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "885573a4a84f44f5b112195de6233c23"
format_stamp: "Formatted at 2026-06-19 14:56:46 on dist-test-slave-76rk"
server_key: "7d62692cc350f8e68f189327252ef5bf"
server_key_iv: "d999d85abf3407ddfc03591e6c4f65ae"
server_key_version: "encryptionkey@0"
I20260619 14:56:46.603264 15560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "885573a4a84f44f5b112195de6233c23"
format_stamp: "Formatted at 2026-06-19 14:56:46 on dist-test-slave-76rk"
server_key: "7d62692cc350f8e68f189327252ef5bf"
server_key_iv: "d999d85abf3407ddfc03591e6c4f65ae"
server_key_version: "encryptionkey@0"
I20260619 14:56:46.608568 15560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 14:56:46.612150 15575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:46.613517 15560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:46.613662 15560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "885573a4a84f44f5b112195de6233c23"
format_stamp: "Formatted at 2026-06-19 14:56:46 on dist-test-slave-76rk"
server_key: "7d62692cc350f8e68f189327252ef5bf"
server_key_iv: "d999d85abf3407ddfc03591e6c4f65ae"
server_key_version: "encryptionkey@0"
I20260619 14:56:46.613831 15560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:46.639753 15560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:46.644466 15560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:46.644731 15560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:46.664165 15560 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:35165
I20260619 14:56:46.664201 15626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:35165 every 8 connection(s)
I20260619 14:56:46.665632 15560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:46.667200 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 15560
I20260619 14:56:46.667460 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:46.668267 11967 external_mini_cluster.cc:1428] Setting key 57484306e97ad2cca532b90d0f04df95
I20260619 14:56:46.670882 15627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 14:56:46 dist-test-slave-76rk krb5kdc[15547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:46.684695 15627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23: Bootstrap starting.
I20260619 14:56:46.688257 15627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:46.689353 15627 log.cc:824] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:46.692188 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:46.670495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36120 (local address 127.11.175.254:35165)
0619 14:56:46.671478 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:46.671490 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:46.671532 (+ 42us) server_negotiation.cc:407] Connection header received
0619 14:56:46.673217 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:46.673244 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:46.673634 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:46.673983 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:46.675909 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:46.677252 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:46.679029 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:46.679702 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:46.679846 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:46.684470 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:46.684479 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:46.684494 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:46.684536 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:46.687857 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:46.688647 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:46.688654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:46.688661 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:46.688843 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:46.689270 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:46.689274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:46.689277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:46.689894 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:46.690236 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:46.690747 (+ 511us) server_negotiation.cc:299] Negotiation successful
0619 14:56:46.691188 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":516,"threads_started":1}
I20260619 14:56:46.692600 15627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23: No bootstrap required, opened a new log
I20260619 14:56:46.700356 15627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885573a4a84f44f5b112195de6233c23" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35165 } }
I20260619 14:56:46.700922 15627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:46.701064 15627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 885573a4a84f44f5b112195de6233c23, State: Initialized, Role: FOLLOWER
I20260619 14:56:46.701785 15627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885573a4a84f44f5b112195de6233c23" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35165 } }
I20260619 14:56:46.702075 15627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:46.702189 15627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:46.702322 15627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:46.704523 15627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885573a4a84f44f5b112195de6233c23" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35165 } }
I20260619 14:56:46.704989 15627 leader_election.cc:302] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 885573a4a84f44f5b112195de6233c23; no voters:
I20260619 14:56:46.705575 15627 leader_election.cc:288] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:46.705734 15632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:46.706439 15632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [term 1 LEADER]: Becoming Leader. State: Replica: 885573a4a84f44f5b112195de6233c23, State: Running, Role: LEADER
I20260619 14:56:46.707062 15632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885573a4a84f44f5b112195de6233c23" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35165 } }
I20260619 14:56:46.707887 15627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:46.711513 15634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "885573a4a84f44f5b112195de6233c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885573a4a84f44f5b112195de6233c23" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35165 } } }
I20260619 14:56:46.711699 15633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 885573a4a84f44f5b112195de6233c23. Latest consensus state: current_term: 1 leader_uuid: "885573a4a84f44f5b112195de6233c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885573a4a84f44f5b112195de6233c23" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35165 } } }
I20260619 14:56:46.712000 15634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:46.712052 15633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:46.714699 15640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:46.720359 15640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:46.729910 15640 catalog_manager.cc:1347] Generated new cluster ID: c95aa35227db411990d4f3750ed77ff1
I20260619 14:56:46.730036 15640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:46.745178 15640 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:46.746558 15640 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:46.755349 15640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 885573a4a84f44f5b112195de6233c23: Generated new TSK 0
I20260619 14:56:46.756310 15640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:56:46.806358 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35165
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:47.145666 15653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:47.145961 15653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:47.146147 15653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:47.153825 15653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:47.153947 15653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:47.154129 15653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:56:47.163913 15653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:47.165267 15653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:47.166584 15653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:47.179687 15658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:47.179873 15659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:47.180339 15661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:47.180830 15653 server_base.cc:1034] running on GCE node
I20260619 14:56:47.181428 15653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:47.182336 15653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:47.183559 15653 hybrid_clock.cc:648] HybridClock initialized: now 1781881007183493 us; error 59 us; skew 500 ppm
I20260619 14:56:47.183933 15653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:47 dist-test-slave-76rk krb5kdc[15547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881007, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:47.187260 15653 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:56:47.188903 15653 webserver.cc:460] Webserver started at http://127.11.175.193:36811/ using document root <none> and password file <none>
I20260619 14:56:47.189649 15653 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:47.189805 15653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:47.190124 15653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:47.193670 15653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "eee1837209074404ae8e2b88f14ce53b"
format_stamp: "Formatted at 2026-06-19 14:56:47 on dist-test-slave-76rk"
server_key: "05238c07bd67b825c094345ecdeeef8f"
server_key_iv: "5653ddc5541f8677ee6eb0c506cd4346"
server_key_version: "encryptionkey@0"
I20260619 14:56:47.194602 15653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "eee1837209074404ae8e2b88f14ce53b"
format_stamp: "Formatted at 2026-06-19 14:56:47 on dist-test-slave-76rk"
server_key: "05238c07bd67b825c094345ecdeeef8f"
server_key_iv: "5653ddc5541f8677ee6eb0c506cd4346"
server_key_version: "encryptionkey@0"
I20260619 14:56:47.200609 15653 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 14:56:47.204712 15668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:47.206194 15653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 14:56:47.206398 15653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "eee1837209074404ae8e2b88f14ce53b"
format_stamp: "Formatted at 2026-06-19 14:56:47 on dist-test-slave-76rk"
server_key: "05238c07bd67b825c094345ecdeeef8f"
server_key_iv: "5653ddc5541f8677ee6eb0c506cd4346"
server_key_version: "encryptionkey@0"
I20260619 14:56:47.206624 15653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:47.226958 15653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:47.231684 15653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:47.232003 15653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:47.233103 15653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:47.234549 15653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:47.234678 15653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:47.234831 15653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:47.234917 15653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:47.269719 15653 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:37751
I20260619 14:56:47.269758 15780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:37751 every 8 connection(s)
I20260619 14:56:47.271106 15653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:56:47.277684 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 15653
I20260619 14:56:47.277925 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:56:47.278545 11967 external_mini_cluster.cc:1428] Setting key 2f09a62d974d920feabe1e74e7c4c5a5
Jun 19 14:56:47 dist-test-slave-76rk krb5kdc[15547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881007, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:47.293484 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:47.273675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:52839 (local address 127.11.175.254:35165)
0619 14:56:47.274142 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:47.274147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:47.275135 (+ 988us) server_negotiation.cc:407] Connection header received
0619 14:56:47.277077 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:47.277083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:47.277213 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:47.277337 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:47.279245 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:47.280223 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:47.282194 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:47.283018 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:47.283168 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:47.286979 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:47.286989 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:47.286996 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:47.287047 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:47.289906 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:47.290997 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:47.291001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:47.291004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:47.291125 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:47.291646 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:47.291650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:47.291652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:47.291925 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:47.292036 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:47.293032 (+ 996us) server_negotiation.cc:299] Negotiation successful
0619 14:56:47.293216 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":266,"threads_started":1}
I20260619 14:56:47.295131 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:47.274154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35165 (local address 127.11.175.193:52839)
0619 14:56:47.274957 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:47.274997 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:47.276753 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:47.277587 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:47.277596 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:47.278298 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:47.279013 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:47.279035 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:47.280452 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:47.280457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:47.282006 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:47.282021 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:47.283276 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:47.283281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:47.283505 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:47.284242 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:47.284270 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:47.286687 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:47.290078 (+ 3391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:47.290090 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:47.290114 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:47.290864 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:47.291237 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:47.291245 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:47.291252 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:47.291531 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:47.292058 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:47.292069 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:47.292577 (+ 508us) client_negotiation.cc:769] Sending connection context
0619 14:56:47.292918 (+ 341us) client_negotiation.cc:240] Negotiation successful
0619 14:56:47.293584 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":265,"threads_started":1}
I20260619 14:56:47.296938 15781 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35165
I20260619 14:56:47.297273 15781 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:47.298065 15781 heartbeater.cc:511] Master 127.11.175.254:35165 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 14:56:47.300345 15592 ts_manager.cc:194] Registered new tserver with Master: eee1837209074404ae8e2b88f14ce53b (127.11.175.193:37751)
I20260619 14:56:47.301838 15592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:52839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:56:47.327946 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35165
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:47.669786 15787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:47.670096 15787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:47.670284 15787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:47.678153 15787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:47.678285 15787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:47.678527 15787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:56:47.689091 15787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:47.690572 15787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:47.692039 15787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:47.705822 15793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:47.706069 15792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:47.706039 15795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:47.707198 15787 server_base.cc:1034] running on GCE node
I20260619 14:56:47.708071 15787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:47.709228 15787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:47.710512 15787 hybrid_clock.cc:648] HybridClock initialized: now 1781881007710454 us; error 48 us; skew 500 ppm
I20260619 14:56:47.710997 15787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:47 dist-test-slave-76rk krb5kdc[15547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881007, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:47.714563 15787 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:56:47.716373 15787 webserver.cc:460] Webserver started at http://127.11.175.194:42211/ using document root <none> and password file <none>
I20260619 14:56:47.717170 15787 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:47.717327 15787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:47.717660 15787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:47.720757 15787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "4d2efde3f7c24436ae8f76b094ca228d"
format_stamp: "Formatted at 2026-06-19 14:56:47 on dist-test-slave-76rk"
server_key: "3a24594dcaca842387a72a304165f6f1"
server_key_iv: "1a42eba408999585b959b4a10ee2c5ea"
server_key_version: "encryptionkey@0"
I20260619 14:56:47.721599 15787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "4d2efde3f7c24436ae8f76b094ca228d"
format_stamp: "Formatted at 2026-06-19 14:56:47 on dist-test-slave-76rk"
server_key: "3a24594dcaca842387a72a304165f6f1"
server_key_iv: "1a42eba408999585b959b4a10ee2c5ea"
server_key_version: "encryptionkey@0"
I20260619 14:56:47.727093 15787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260619 14:56:47.730885 15802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:47.732371 15787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:47.732586 15787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "4d2efde3f7c24436ae8f76b094ca228d"
format_stamp: "Formatted at 2026-06-19 14:56:47 on dist-test-slave-76rk"
server_key: "3a24594dcaca842387a72a304165f6f1"
server_key_iv: "1a42eba408999585b959b4a10ee2c5ea"
server_key_version: "encryptionkey@0"
I20260619 14:56:47.732807 15787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:47.755578 15787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:47.760164 15787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:47.760428 15787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:47.761368 15787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:47.762768 15787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:47.762858 15787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:47.762946 15787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:47.762991 15787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:47.797616 15787 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:34571
I20260619 14:56:47.797703 15914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:34571 every 8 connection(s)
I20260619 14:56:47.798985 15787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:56:47.799659 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 15787
I20260619 14:56:47.799871 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:56:47.800436 11967 external_mini_cluster.cc:1428] Setting key 100e7367e0e0ae09ad8d001a6b4fdcdb
Jun 19 14:56:47 dist-test-slave-76rk krb5kdc[15547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881007, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:47.822144 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:47.801616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:36859 (local address 127.11.175.254:35165)
0619 14:56:47.802079 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:47.802085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:47.803152 (+ 1067us) server_negotiation.cc:407] Connection header received
0619 14:56:47.805131 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:47.805137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:47.805242 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:47.805363 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:47.807097 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:47.807848 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:47.809723 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:47.810168 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:47.810278 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:47.815175 (+ 4897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:47.815180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:47.815183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:47.815217 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:47.818281 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:47.819488 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:47.819492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:47.819494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:47.819629 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:47.820183 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:47.820186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:47.820188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:47.820482 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:47.820608 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:47.821660 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0619 14:56:47.821852 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":221,"threads_started":1}
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
I20260619 14:56:47.823725 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:47.802205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35165 (local address 127.11.175.194:36859)
0619 14:56:47.802998 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:47.803036 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:47.804837 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:47.805635 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:47.805645 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:47.806204 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:47.806904 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:47.806925 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:47.808034 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:47.808038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:47.809562 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:47.809581 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:47.810421 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:47.810431 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:47.810843 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:47.811659 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:47.811692 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:47.814907 (+ 3215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:47.818459 (+ 3552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:47.818473 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:47.818519 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:47.819325 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:47.819751 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:47.819758 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:47.819768 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:47.820062 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:47.820639 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:47.820649 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:47.821176 (+ 527us) client_negotiation.cc:769] Sending connection context
0619 14:56:47.821539 (+ 363us) client_negotiation.cc:240] Negotiation successful
0619 14:56:47.822206 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":310,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
I20260619 14:56:47.825390 15915 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35165
I20260619 14:56:47.825750 15915 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:47.826534 15915 heartbeater.cc:511] Master 127.11.175.254:35165 requested a full tablet report, sending...
I20260619 14:56:47.828363 15592 ts_manager.cc:194] Registered new tserver with Master: 4d2efde3f7c24436ae8f76b094ca228d (127.11.175.194:34571)
I20260619 14:56:47.829679 15592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:36859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:56:47.851768 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35165
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:48.182013 15921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:48.182363 15921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:48.182502 15921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:48.190063 15921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:48.190250 15921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:48.190441 15921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:56:48.200675 15921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:48.202057 15921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:48.203404 15921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:48.217644 15926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:48.217629 15927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:48.217763 15929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:48.220018 15921 server_base.cc:1034] running on GCE node
I20260619 14:56:48.220636 15921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:48.221702 15921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:48.222970 15921 hybrid_clock.cc:648] HybridClock initialized: now 1781881008222943 us; error 56 us; skew 500 ppm
I20260619 14:56:48.223304 15921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:48 dist-test-slave-76rk krb5kdc[15547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:48.226595 15921 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:56:48.228227 15921 webserver.cc:460] Webserver started at http://127.11.175.195:40347/ using document root <none> and password file <none>
I20260619 14:56:48.228969 15921 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:48.229113 15921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:48.229432 15921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:48.232424 15921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "c09f89b8332b4cab8030afa037c4e68f"
format_stamp: "Formatted at 2026-06-19 14:56:48 on dist-test-slave-76rk"
server_key: "5ec064526257019875e8d54fc81c5c5a"
server_key_iv: "51916d6849b7eea58d395813548a59ba"
server_key_version: "encryptionkey@0"
I20260619 14:56:48.233234 15921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "c09f89b8332b4cab8030afa037c4e68f"
format_stamp: "Formatted at 2026-06-19 14:56:48 on dist-test-slave-76rk"
server_key: "5ec064526257019875e8d54fc81c5c5a"
server_key_iv: "51916d6849b7eea58d395813548a59ba"
server_key_version: "encryptionkey@0"
I20260619 14:56:48.239128 15921 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 14:56:48.242868 15936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:48.244450 15921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:48.244673 15921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "c09f89b8332b4cab8030afa037c4e68f"
format_stamp: "Formatted at 2026-06-19 14:56:48 on dist-test-slave-76rk"
server_key: "5ec064526257019875e8d54fc81c5c5a"
server_key_iv: "51916d6849b7eea58d395813548a59ba"
server_key_version: "encryptionkey@0"
I20260619 14:56:48.244920 15921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:48.260099 15921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:48.265465 15921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:48.265844 15921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:48.267060 15921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:48.268673 15921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:48.268770 15921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:48.268867 15921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:48.268940 15921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:48.305950 15781 heartbeater.cc:503] Master 127.11.175.254:35165 was elected leader, sending a full tablet report...
I20260619 14:56:48.308550 15921 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:38989
I20260619 14:56:48.308585 16048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:38989 every 8 connection(s)
I20260619 14:56:48.309952 15921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:56:48.313524 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 15921
I20260619 14:56:48.313750 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:56:48.314416 11967 external_mini_cluster.cc:1428] Setting key 74ea4e78487d2bb25fc2ff65e2367670
Jun 19 14:56:48 dist-test-slave-76rk krb5kdc[15547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:48.332885 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:48.312759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:37035 (local address 127.11.175.254:35165)
0619 14:56:48.312941 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:48.312946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:48.314437 (+ 1491us) server_negotiation.cc:407] Connection header received
0619 14:56:48.316693 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:48.316698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:48.316820 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:48.316947 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:48.319174 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.319941 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.321804 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.322218 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.322332 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:48.326497 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:48.326502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:48.326506 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:48.326543 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:48.329595 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.330720 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.330727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.330733 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.330904 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.331353 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.331357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.331359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.331629 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:48.331750 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:48.332471 (+ 721us) server_negotiation.cc:299] Negotiation successful
0619 14:56:48.332642 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 14:56:48.333843 16051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:48.313391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35165 (local address 127.11.175.195:37035)
0619 14:56:48.314269 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:48.314311 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:48.316419 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:48.317288 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:48.317298 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:48.318180 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:48.319000 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:48.319024 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.320128 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.320133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:48.321634 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:48.321649 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.322456 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.322462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:48.322806 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:48.323606 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:48.323632 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:48.326225 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:48.329753 (+ 3528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:48.329760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:48.329775 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:48.330411 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:48.331031 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:48.331036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:48.331039 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:48.331245 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:48.331733 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:48.331738 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:48.332117 (+ 379us) client_negotiation.cc:769] Sending connection context
0619 14:56:48.332344 (+ 227us) client_negotiation.cc:240] Negotiation successful
0619 14:56:48.332763 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":410,"threads_started":1}
I20260619 14:56:48.335340 16049 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35165
I20260619 14:56:48.335711 16049 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:48.336536 16049 heartbeater.cc:511] Master 127.11.175.254:35165 requested a full tablet report, sending...
I20260619 14:56:48.338106 15592 ts_manager.cc:194] Registered new tserver with Master: c09f89b8332b4cab8030afa037c4e68f (127.11.175.195:38989)
I20260619 14:56:48.338972 15592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:37035
I20260619 14:56:48.343873 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 14:56:48 dist-test-slave-76rk krb5kdc[15547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 14:56:48 dist-test-slave-76rk krb5kdc[15547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:56:48.398792 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:48.380993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.11.175.193:37751)
0619 14:56:48.381830 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:48.381838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:48.381866 (+ 28us) server_negotiation.cc:407] Connection header received
0619 14:56:48.381927 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:48.381933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:48.382161 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:48.382303 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:48.383930 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.385334 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.386616 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.387355 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.387469 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:48.392093 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:48.392109 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:48.392126 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:48.392170 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:48.395609 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.396392 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.396400 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.396402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.396541 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.396956 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.396960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.396962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.397327 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:48.397563 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:48.398119 (+ 556us) server_negotiation.cc:299] Negotiation successful
0619 14:56:48.398381 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":557,"threads_started":1}
Jun 19 14:56:48 dist-test-slave-76rk krb5kdc[15547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:48.419313 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:48.403758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.11.175.254:35165)
0619 14:56:48.404060 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:48.404068 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:48.404090 (+ 22us) server_negotiation.cc:407] Connection header received
0619 14:56:48.404298 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:48.404303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:48.404388 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:48.404501 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:48.405908 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.406722 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.408141 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.408633 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.408757 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:48.413641 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:48.413646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:48.413650 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:48.413686 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:48.416570 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.417271 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.417275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.417277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.417412 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.417776 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.417779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.417781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.418130 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:48.418335 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:48.418739 (+ 404us) server_negotiation.cc:299] Negotiation successful
0619 14:56:48.418952 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260619 14:56:48.419976 15592 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36136
I20260619 14:56:48.421159 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15653
I20260619 14:56:48.434762 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:48.422410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52790 (local address 127.11.175.193:37751)
0619 14:56:48.422696 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:48.422703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:48.422777 (+ 74us) server_negotiation.cc:407] Connection header received
0619 14:56:48.422968 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:48.422972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:48.423066 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:48.423155 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:48.424585 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.425332 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.426628 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.427111 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.427211 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:48.429645 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:48.429650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:48.429653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:48.429687 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:48.432436 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.433083 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.433087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.433089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.433213 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.433557 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.433560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.433562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.433797 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:48.433975 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:48.434263 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 14:56:48.434429 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260619 14:56:48.435504 15776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:48.519615 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 15653
I20260619 14:56:48.536154 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15787
Jun 19 14:56:48 dist-test-slave-76rk krb5kdc[15547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:56:48.556818 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:48.537115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40646 (local address 127.11.175.194:34571)
0619 14:56:48.537671 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:48.537678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:48.537705 (+ 27us) server_negotiation.cc:407] Connection header received
0619 14:56:48.537768 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:48.537775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:48.538030 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:48.538166 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:48.539519 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.540531 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.541764 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.542339 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.542437 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:48.550518 (+ 8081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:48.550533 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:48.550548 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:48.550599 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:48.553781 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.554436 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.554443 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.554445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.554576 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.555171 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.555175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.555177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.555562 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:48.555814 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:48.556165 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 14:56:48.556458 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":288,"threads_started":1}
I20260619 14:56:48.558133 15910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:48.642561 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 15787
I20260619 14:56:48.660045 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15921
Jun 19 14:56:48 dist-test-slave-76rk krb5kdc[15547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:56:48.677973 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:48.661073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.11.175.195:38989)
0619 14:56:48.661702 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:48.661709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:48.661738 (+ 29us) server_negotiation.cc:407] Connection header received
0619 14:56:48.661797 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:48.661803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:48.662037 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:48.662162 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:48.663580 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.665287 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.666691 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:48.667595 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:48.667727 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:48.672232 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:48.672244 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:48.672258 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:48.672294 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:48.675291 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.675988 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.675995 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.675997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.676121 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:48.676468 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:48.676472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:48.676474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:48.676821 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:48.677092 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:48.677307 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 14:56:48.677521 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":315,"threads_started":1}
I20260619 14:56:48.679546 16044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:48.763286 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 15921
I20260619 14:56:48.780035 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 15560
I20260619 14:56:48.781364 15622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:48.890866 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 15560
2026-06-19T14:56:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4010 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:56:48 dist-test-slave-76rk krb5kdc[16068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:56:48 dist-test-slave-76rk krb5kdc[16068](info): set up 2 sockets
Jun 19 14:56:48 dist-test-slave-76rk krb5kdc[16068](info): commencing operation
krb5kdc: starting...
W20260619 14:56:49.995199 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:56:50 dist-test-slave-76rk krb5kdc[16068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:56:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:56:50.214138 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:42691
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:42691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:50.553470 16081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:50.553756 16081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:50.553920 16081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:50.561383 16081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:50.561501 16081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:50.561586 16081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:50.561689 16081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:50.561786 16081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:56:50.571830 16081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:42691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:42691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:50.573163 16081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:50.574535 16081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:50.586144 16086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:50.586180 16089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:50.586211 16087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:50.587008 16081 server_base.cc:1034] running on GCE node
I20260619 14:56:50.587807 16081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:50.589300 16081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:50.590546 16081 hybrid_clock.cc:648] HybridClock initialized: now 1781881010590507 us; error 63 us; skew 500 ppm
I20260619 14:56:50.590932 16081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:50 dist-test-slave-76rk krb5kdc[16068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881010, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:50.594244 16081 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:56:50.595580 16081 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:50.595721 16081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:50.596065 16081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:50.599184 16081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0"
format_stamp: "Formatted at 2026-06-19 14:56:50 on dist-test-slave-76rk"
server_key: "6240d7c84da6fbd492c0fd4efbb6c09f"
server_key_iv: "6dbe408e9070870dc31b6f10e0d1908f"
server_key_version: "encryptionkey@0"
I20260619 14:56:50.600010 16081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0"
format_stamp: "Formatted at 2026-06-19 14:56:50 on dist-test-slave-76rk"
server_key: "6240d7c84da6fbd492c0fd4efbb6c09f"
server_key_iv: "6dbe408e9070870dc31b6f10e0d1908f"
server_key_version: "encryptionkey@0"
I20260619 14:56:50.605405 16081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:56:50.608937 16095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:50.610283 16081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 14:56:50.610484 16081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0"
format_stamp: "Formatted at 2026-06-19 14:56:50 on dist-test-slave-76rk"
server_key: "6240d7c84da6fbd492c0fd4efbb6c09f"
server_key_iv: "6dbe408e9070870dc31b6f10e0d1908f"
server_key_version: "encryptionkey@0"
I20260619 14:56:50.610752 16081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:50.626744 16081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:50.631273 16081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:50.631546 16081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:50.650476 16081 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:42691
I20260619 14:56:50.650497 16146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:42691 every 8 connection(s)
I20260619 14:56:50.651437 16081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:50.655164 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 16081
I20260619 14:56:50.655292 16147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:50.655409 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:50.656231 11967 external_mini_cluster.cc:1428] Setting key 486afde2678cd1feb8ead764d19ceab5
I20260619 14:56:50.670048 16147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0: Bootstrap starting.
Jun 19 14:56:50 dist-test-slave-76rk krb5kdc[16068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:50.673267 16147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:50.674685 16147 log.cc:824] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:50.678081 16147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0: No bootstrap required, opened a new log
I20260619 14:56:50.682130 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:50.658424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45878 (local address 127.11.175.254:42691)
0619 14:56:50.659286 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:50.659302 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:50.659351 (+ 49us) server_negotiation.cc:407] Connection header received
0619 14:56:50.662286 (+ 2935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:50.662317 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:50.662925 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:50.663404 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:50.664793 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:50.666825 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:50.668105 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:50.669159 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:50.669328 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:50.673387 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:50.673398 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:50.673418 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:50.673472 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:50.677454 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:50.678144 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:50.678153 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:50.678161 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:50.678369 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:50.678800 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:50.678809 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:50.678813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:50.679552 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:50.679844 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:50.680214 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 14:56:50.680688 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":389,"threads_started":1}
I20260619 14:56:50.687629 16147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42691 } }
I20260619 14:56:50.687964 16147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:50.688041 16147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0f5ee1fd4c4b34b622cee13b4860a0, State: Initialized, Role: FOLLOWER
I20260619 14:56:50.688532 16147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42691 } }
I20260619 14:56:50.688724 16147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:50.688817 16147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:50.688937 16147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:50.691255 16147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42691 } }
I20260619 14:56:50.691856 16147 leader_election.cc:302] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f0f5ee1fd4c4b34b622cee13b4860a0; no voters:
I20260619 14:56:50.692476 16147 leader_election.cc:288] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:50.692627 16152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:50.693439 16152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [term 1 LEADER]: Becoming Leader. State: Replica: 7f0f5ee1fd4c4b34b622cee13b4860a0, State: Running, Role: LEADER
I20260619 14:56:50.693967 16152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42691 } }
I20260619 14:56:50.694880 16147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:50.696973 16154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f0f5ee1fd4c4b34b622cee13b4860a0. Latest consensus state: current_term: 1 leader_uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42691 } } }
I20260619 14:56:50.697165 16153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0f5ee1fd4c4b34b622cee13b4860a0" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42691 } } }
I20260619 14:56:50.697438 16154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:50.697465 16153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:50.699172 16159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:50.704377 16159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:50.713476 16159 catalog_manager.cc:1347] Generated new cluster ID: 74005b8a610444b3a91b62b34d8718f8
I20260619 14:56:50.713603 16159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:50.720433 16159 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:50.721709 16159 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:50.731882 16159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0: Generated new TSK 0
I20260619 14:56:50.732771 16159 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:56:50.786438 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42691
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:51.134377 16173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:51.134799 16173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:51.134967 16173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:51.142300 16173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:51.142422 16173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:51.142539 16173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:56:51.152486 16173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:51.153735 16173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:51.155081 16173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:51.167922 16178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:51.168020 16179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:51.167951 16181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:51.168540 16173 server_base.cc:1034] running on GCE node
I20260619 14:56:51.169137 16173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:51.170032 16173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:51.171280 16173 hybrid_clock.cc:648] HybridClock initialized: now 1781881011171213 us; error 34 us; skew 500 ppm
I20260619 14:56:51.171571 16173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:51 dist-test-slave-76rk krb5kdc[16068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:51.174854 16173 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:56:51.176071 16173 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:51.176201 16173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:51.176576 16173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:51.179490 16173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "aad16fdbf1ce48f0a24a8cb149480425"
format_stamp: "Formatted at 2026-06-19 14:56:51 on dist-test-slave-76rk"
server_key: "cf09902f957acf2df5cca7b825668bbd"
server_key_iv: "ba741958f579c06dd9a4f474ab2d736e"
server_key_version: "encryptionkey@0"
I20260619 14:56:51.180228 16173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "aad16fdbf1ce48f0a24a8cb149480425"
format_stamp: "Formatted at 2026-06-19 14:56:51 on dist-test-slave-76rk"
server_key: "cf09902f957acf2df5cca7b825668bbd"
server_key_iv: "ba741958f579c06dd9a4f474ab2d736e"
server_key_version: "encryptionkey@0"
I20260619 14:56:51.185151 16173 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 14:56:51.188802 16187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:51.190397 16173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 14:56:51.190680 16173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "aad16fdbf1ce48f0a24a8cb149480425"
format_stamp: "Formatted at 2026-06-19 14:56:51 on dist-test-slave-76rk"
server_key: "cf09902f957acf2df5cca7b825668bbd"
server_key_iv: "ba741958f579c06dd9a4f474ab2d736e"
server_key_version: "encryptionkey@0"
I20260619 14:56:51.190970 16173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:51.206368 16173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:51.211040 16173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:51.211329 16173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:51.212320 16173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:51.213725 16173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:51.213851 16173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:51.214001 16173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:51.214099 16173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:51.250056 16173 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:43103
I20260619 14:56:51.250118 16299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:43103 every 8 connection(s)
I20260619 14:56:51.251041 16173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:56:51.258137 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 16173
I20260619 14:56:51.258363 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:56:51.259156 11967 external_mini_cluster.cc:1428] Setting key e523ba05bf50e507dfe68d920f4ca197
Jun 19 14:56:51 dist-test-slave-76rk krb5kdc[16068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
I20260619 14:56:51.280817 16302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:51.253698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:51691 (local address 127.11.175.254:42691)
0619 14:56:51.254205 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:51.254212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:51.255349 (+ 1137us) server_negotiation.cc:407] Connection header received
0619 14:56:51.257177 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:51.257182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:51.257250 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:51.257350 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:51.259201 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:51.260145 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:51.261976 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:51.262367 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:51.262517 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:51.266185 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:51.266190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:51.266193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:51.266231 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:51.276391 (+ 10160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:51.277737 (+ 1346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:51.277742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:51.277745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:51.277900 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:51.278523 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:51.278528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:51.278539 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:51.278943 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:51.279082 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:51.280148 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0619 14:56:51.280386 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":249,"threads_started":1}
I20260619 14:56:51.281876 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:51.254357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42691 (local address 127.11.175.193:51691)
0619 14:56:51.255176 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:51.255220 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:51.256903 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:51.257566 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:51.257574 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:51.258147 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:51.259008 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:51.259029 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:51.260341 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:51.260348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:51.261791 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:51.261805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:51.262615 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:51.262624 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:51.262893 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:51.263652 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:51.263681 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:51.265921 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:51.276651 (+ 10730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:51.276666 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:51.276690 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:51.277529 (+ 839us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:51.278034 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:51.278040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:51.278044 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:51.278365 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:51.279181 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:51.279188 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:51.279583 (+ 395us) client_negotiation.cc:769] Sending connection context
0619 14:56:51.279848 (+ 265us) client_negotiation.cc:240] Negotiation successful
0619 14:56:51.280338 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":334,"threads_started":1}
I20260619 14:56:51.283802 16300 heartbeater.cc:347] Connected to a master server at 127.11.175.254:42691
I20260619 14:56:51.284209 16300 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:51.285243 16300 heartbeater.cc:511] Master 127.11.175.254:42691 requested a full tablet report, sending...
I20260619 14:56:51.288025 16112 ts_manager.cc:194] Registered new tserver with Master: aad16fdbf1ce48f0a24a8cb149480425 (127.11.175.193:43103)
I20260619 14:56:51.289740 16112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:51691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:56:51.310312 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42691
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:51.635200 16306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:51.635655 16306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:51.635846 16306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:51.643996 16306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:51.644114 16306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:51.644238 16306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:56:51.654165 16306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:51.655535 16306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:51.656798 16306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:51.669775 16311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:51.669983 16312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:51.669862 16314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:51.671350 16306 server_base.cc:1034] running on GCE node
I20260619 14:56:51.671933 16306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:51.672971 16306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:51.674198 16306 hybrid_clock.cc:648] HybridClock initialized: now 1781881011674137 us; error 48 us; skew 500 ppm
I20260619 14:56:51.674522 16306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:51 dist-test-slave-76rk krb5kdc[16068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:51.677898 16306 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:56:51.679191 16306 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:51.679281 16306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:51.679613 16306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:51.682541 16306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "8e151c4f810f4ef48052f8d57aa4cf15"
format_stamp: "Formatted at 2026-06-19 14:56:51 on dist-test-slave-76rk"
server_key: "753873721a28c8352ff8509f45d718e1"
server_key_iv: "938e4a29bb1a50fa486cd57dd00a6bc4"
server_key_version: "encryptionkey@0"
I20260619 14:56:51.683425 16306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "8e151c4f810f4ef48052f8d57aa4cf15"
format_stamp: "Formatted at 2026-06-19 14:56:51 on dist-test-slave-76rk"
server_key: "753873721a28c8352ff8509f45d718e1"
server_key_iv: "938e4a29bb1a50fa486cd57dd00a6bc4"
server_key_version: "encryptionkey@0"
I20260619 14:56:51.688544 16306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:56:51.692320 16320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:51.693748 16306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 14:56:51.693931 16306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "8e151c4f810f4ef48052f8d57aa4cf15"
format_stamp: "Formatted at 2026-06-19 14:56:51 on dist-test-slave-76rk"
server_key: "753873721a28c8352ff8509f45d718e1"
server_key_iv: "938e4a29bb1a50fa486cd57dd00a6bc4"
server_key_version: "encryptionkey@0"
I20260619 14:56:51.694180 16306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:51.713606 16306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:51.718108 16306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:51.718336 16306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:51.719275 16306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:51.720589 16306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:51.720674 16306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:51.720755 16306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:51.720809 16306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:51.753194 16306 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:45059
I20260619 14:56:51.753244 16432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:45059 every 8 connection(s)
I20260619 14:56:51.754094 16306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:56:51.761097 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 16306
I20260619 14:56:51.761292 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:56:51.761734 11967 external_mini_cluster.cc:1428] Setting key 5f1259583002e21f05d27ab56ffd32cb
Jun 19 14:56:51 dist-test-slave-76rk krb5kdc[16068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:51.775153 16302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:51.756696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:42871 (local address 127.11.175.254:42691)
0619 14:56:51.756884 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:51.756889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:51.758063 (+ 1174us) server_negotiation.cc:407] Connection header received
0619 14:56:51.760092 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:51.760098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:51.760193 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:51.760306 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:51.761981 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:51.762900 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:51.764648 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:51.765091 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:51.765221 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:51.768889 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:51.768899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:51.768908 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:51.768965 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:51.771980 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:51.772757 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:51.772763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:51.772767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:51.772901 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:51.773330 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:51.773336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:51.773339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:51.773621 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:51.773782 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:51.774506 (+ 724us) server_negotiation.cc:299] Negotiation successful
0619 14:56:51.774793 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 14:56:51.776173 16435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:51.757210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42691 (local address 127.11.175.194:42871)
0619 14:56:51.757896 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:51.757933 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:51.759743 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:51.760557 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:51.760568 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:51.761085 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:51.761762 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:51.761804 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:51.763049 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:51.763055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:51.764493 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:51.764505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:51.765318 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:51.765325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:51.765548 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:51.766225 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:51.766248 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:51.768641 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:51.772142 (+ 3501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:51.772151 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:51.772169 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:51.772618 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:51.773022 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:51.773027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:51.773031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:51.773216 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:51.773756 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:51.773765 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:51.774125 (+ 360us) client_negotiation.cc:769] Sending connection context
0619 14:56:51.774400 (+ 275us) client_negotiation.cc:240] Negotiation successful
0619 14:56:51.775023 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":279,"threads_started":1}
I20260619 14:56:51.777910 16433 heartbeater.cc:347] Connected to a master server at 127.11.175.254:42691
I20260619 14:56:51.778254 16433 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:51.778970 16433 heartbeater.cc:511] Master 127.11.175.254:42691 requested a full tablet report, sending...
I20260619 14:56:51.780612 16112 ts_manager.cc:194] Registered new tserver with Master: 8e151c4f810f4ef48052f8d57aa4cf15 (127.11.175.194:45059)
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
I20260619 14:56:51.781800 16112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:42871
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:56:51.807803 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42691
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:56:52.137871 16438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:52.138173 16438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:52.138346 16438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:52.145759 16438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:52.145875 16438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:52.146063 16438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:56:52.156306 16438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:52.157655 16438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:52.158968 16438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:52.172191 16443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:52.172228 16446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:52.172266 16444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:52.173252 16438 server_base.cc:1034] running on GCE node
I20260619 14:56:52.173863 16438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:52.174909 16438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:52.176122 16438 hybrid_clock.cc:648] HybridClock initialized: now 1781881012176066 us; error 46 us; skew 500 ppm
I20260619 14:56:52.176465 16438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:56:52 dist-test-slave-76rk krb5kdc[16068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881012, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:56:52.179708 16438 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:56:52.180914 16438 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:52.181012 16438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:52.181306 16438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:52.184259 16438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "3fc52e9e52d741e0a180eb7d80089255"
format_stamp: "Formatted at 2026-06-19 14:56:52 on dist-test-slave-76rk"
server_key: "ad312582c570a9354e9dcfdf86ce7c90"
server_key_iv: "41e0c9452b711d7b1a14f9a875f3cc37"
server_key_version: "encryptionkey@0"
I20260619 14:56:52.185024 16438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "3fc52e9e52d741e0a180eb7d80089255"
format_stamp: "Formatted at 2026-06-19 14:56:52 on dist-test-slave-76rk"
server_key: "ad312582c570a9354e9dcfdf86ce7c90"
server_key_iv: "41e0c9452b711d7b1a14f9a875f3cc37"
server_key_version: "encryptionkey@0"
I20260619 14:56:52.190157 16438 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 14:56:52.193887 16452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:52.195408 16438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:56:52.195559 16438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "3fc52e9e52d741e0a180eb7d80089255"
format_stamp: "Formatted at 2026-06-19 14:56:52 on dist-test-slave-76rk"
server_key: "ad312582c570a9354e9dcfdf86ce7c90"
server_key_iv: "41e0c9452b711d7b1a14f9a875f3cc37"
server_key_version: "encryptionkey@0"
I20260619 14:56:52.195741 16438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:52.218247 16438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:52.222535 16438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:52.222859 16438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:52.223824 16438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:52.225174 16438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:52.225262 16438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:52.225337 16438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:52.225391 16438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:52.259198 16438 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:34709
I20260619 14:56:52.259328 16564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:34709 every 8 connection(s)
I20260619 14:56:52.260061 16438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:56:52.269291 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 16438
I20260619 14:56:52.269492 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:56:52.269943 11967 external_mini_cluster.cc:1428] Setting key 871b0fa8ef5a831f64b7e5f5ace456ba
Jun 19 14:56:52 dist-test-slave-76rk krb5kdc[16068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881012, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:56:52.280336 16302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.262748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:53157 (local address 127.11.175.254:42691)
0619 14:56:52.263035 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.263039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.264070 (+ 1031us) server_negotiation.cc:407] Connection header received
0619 14:56:52.265908 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.265913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.265988 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.266079 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:52.267705 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.268437 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.270167 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.270745 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.270882 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.274698 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:52.274704 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:52.274707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:52.274741 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:52.277202 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:52.277972 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:52.277978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:52.277981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:52.278110 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:52.278532 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:52.278539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:52.278544 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:52.278898 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:52.279058 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.279756 (+ 698us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.279985 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260619 14:56:52.281319 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.263171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42691 (local address 127.11.175.195:53157)
0619 14:56:52.263868 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:52.263905 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:52.265618 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:52.266305 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:52.266314 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:52.266929 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:52.267521 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.267539 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.268572 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.268576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.270008 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.270025 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.270996 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.271004 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.271231 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.272072 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:56:52.272104 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:52.274449 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:52.277360 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:52.277369 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:52.277387 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:52.277833 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:52.278223 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:56:52.278228 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:56:52.278232 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:56:52.278418 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:56:52.279030 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:52.279037 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:52.279379 (+ 342us) client_negotiation.cc:769] Sending connection context
0619 14:56:52.279645 (+ 266us) client_negotiation.cc:240] Negotiation successful
0619 14:56:52.280198 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":278,"threads_started":1}
I20260619 14:56:52.282949 16565 heartbeater.cc:347] Connected to a master server at 127.11.175.254:42691
I20260619 14:56:52.283274 16565 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:52.284107 16565 heartbeater.cc:511] Master 127.11.175.254:42691 requested a full tablet report, sending...
I20260619 14:56:52.285941 16112 ts_manager.cc:194] Registered new tserver with Master: 3fc52e9e52d741e0a180eb7d80089255 (127.11.175.195:34709)
I20260619 14:56:52.286849 16112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:53157
I20260619 14:56:52.293817 16300 heartbeater.cc:503] Master 127.11.175.254:42691 was elected leader, sending a full tablet report...
I20260619 14:56:52.298241 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 14:56:52.315892 16302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.303485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59372 (local address 127.11.175.254:42691)
0619 14:56:52.303728 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.303739 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.304169 (+ 430us) server_negotiation.cc:407] Connection header received
0619 14:56:52.304309 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.304313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.304378 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.304475 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:52.305894 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.306623 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.307918 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.308318 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.308412 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.310780 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:52.310793 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:52.310797 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:52.310832 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:52.313252 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:52.313885 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:52.313889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:52.313892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:52.314035 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:52.314439 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:52.314442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:52.314444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:52.314863 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:52.315083 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.315483 (+ 400us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.315670 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 14:56:52.320317 16112 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:56:52.327348 16112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:52.361441 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.339891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47782 (local address 127.11.175.193:43103)
0619 14:56:52.352953 (+ 13062us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.352965 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.353000 (+ 35us) server_negotiation.cc:407] Connection header received
0619 14:56:52.353115 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.353126 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.353521 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.353790 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:52.354778 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.356304 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.359063 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.360162 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.360372 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.360553 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.360903 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.361084 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12866,"thread_start_us":14149,"threads_started":1}
I20260619 14:56:52.361809 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.339352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43103 (local address 127.0.0.1:47782)
0619 14:56:52.340283 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:52.340334 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:52.340550 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:52.353818 (+ 13268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:52.353829 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:52.353866 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:52.354534 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.354554 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.356455 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.356463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.358904 (+ 2441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.358916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.360371 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.360378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.360615 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.360691 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 14:56:52.360833 (+ 142us) client_negotiation.cc:240] Negotiation successful
0619 14:56:52.361019 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":136,"spinlock_wait_cycles":13440,"thread_start_us":755,"threads_started":1}
I20260619 14:56:52.362015 16578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.339547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:34709 (local address 127.0.0.1:49276)
0619 14:56:52.341161 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:52.341177 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:52.341275 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:52.353974 (+ 12699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:52.353985 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:52.354016 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:52.354581 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.354593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.356569 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.356574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.358956 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.358971 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.360663 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.360670 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.360829 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.360857 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 14:56:52.360909 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 14:56:52.361019 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1498,"mutex_wait_us":73,"spinlock_wait_cycles":1792,"thread_start_us":1907,"threads_started":1}
I20260619 14:56:52.362545 16581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.341863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:45059 (local address 127.0.0.1:43716)
0619 14:56:52.352937 (+ 11074us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:52.352959 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:52.353090 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:52.355302 (+ 2212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:52.355306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:52.355330 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:52.355693 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.355704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.357632 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.357637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.359748 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.359762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.361775 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.361780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.361948 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.361990 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 14:56:52.362076 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 14:56:52.362186 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10905,"mutex_wait_us":1,"thread_start_us":10681,"threads_started":1}
I20260619 14:56:52.363112 16582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.342137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.11.175.194:45059)
0619 14:56:52.354725 (+ 12588us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.354734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.354762 (+ 28us) server_negotiation.cc:407] Connection header received
0619 14:56:52.354855 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.354863 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.355091 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.355287 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:52.355876 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.357452 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.359925 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.361256 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.361432 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.361627 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.362522 (+ 895us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.362738 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12410,"thread_start_us":12262,"threads_started":1}
I20260619 14:56:52.363144 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.339868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49276 (local address 127.11.175.195:34709)
0619 14:56:52.353308 (+ 13440us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.353316 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.353351 (+ 35us) server_negotiation.cc:407] Connection header received
0619 14:56:52.353458 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.353467 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.353733 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.354783 (+ 1050us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:52.355034 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.356422 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.359139 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.360444 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.361302 (+ 858us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.362545 (+ 1243us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.362606 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.362789 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13251,"thread_start_us":13569,"threads_started":1}
I20260619 14:56:52.366372 16235 tablet_service.cc:1467] Processing CreateTablet for tablet dd2c4f8d0d0547ebbc8adae7b2c84750 (DEFAULT_TABLE table=test-table [id=41bacc35796e43ef8a31fee502ace5cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:52.366626 16500 tablet_service.cc:1467] Processing CreateTablet for tablet dd2c4f8d0d0547ebbc8adae7b2c84750 (DEFAULT_TABLE table=test-table [id=41bacc35796e43ef8a31fee502ace5cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:52.366910 16368 tablet_service.cc:1467] Processing CreateTablet for tablet dd2c4f8d0d0547ebbc8adae7b2c84750 (DEFAULT_TABLE table=test-table [id=41bacc35796e43ef8a31fee502ace5cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:52.367888 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd2c4f8d0d0547ebbc8adae7b2c84750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:52.367978 16235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd2c4f8d0d0547ebbc8adae7b2c84750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:52.368098 16368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd2c4f8d0d0547ebbc8adae7b2c84750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:52.379179 16583 tablet_bootstrap.cc:492] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: Bootstrap starting.
I20260619 14:56:52.382808 16585 tablet_bootstrap.cc:492] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: Bootstrap starting.
I20260619 14:56:52.384094 16583 tablet_bootstrap.cc:654] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:52.385524 16583 log.cc:824] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:52.386600 16585 tablet_bootstrap.cc:654] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:52.387161 16584 tablet_bootstrap.cc:492] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: Bootstrap starting.
I20260619 14:56:52.387959 16585 log.cc:824] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:52.389634 16583 tablet_bootstrap.cc:492] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: No bootstrap required, opened a new log
I20260619 14:56:52.389937 16583 ts_tablet_manager.cc:1397] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260619 14:56:52.391291 16585 tablet_bootstrap.cc:492] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: No bootstrap required, opened a new log
I20260619 14:56:52.391515 16584 tablet_bootstrap.cc:654] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:52.391574 16585 ts_tablet_manager.cc:1397] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260619 14:56:52.392602 16584 log.cc:824] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:52.395566 16584 tablet_bootstrap.cc:492] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: No bootstrap required, opened a new log
I20260619 14:56:52.395879 16584 ts_tablet_manager.cc:1397] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260619 14:56:52.401844 16583 raft_consensus.cc:348] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 } } peers { permanent_uuid: "3fc52e9e52d741e0a180eb7d80089255" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 34709 } } peers { permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 } }
I20260619 14:56:52.402336 16583 raft_consensus.cc:374] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:52.402458 16583 raft_consensus.cc:729] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc52e9e52d741e0a180eb7d80089255, State: Initialized, Role: FOLLOWER
I20260619 14:56:52.402618 16585 raft_consensus.cc:348] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 } } peers { permanent_uuid: "3fc52e9e52d741e0a180eb7d80089255" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 34709 } } peers { permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 } }
I20260619 14:56:52.403081 16585 raft_consensus.cc:374] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:52.403188 16585 raft_consensus.cc:729] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad16fdbf1ce48f0a24a8cb149480425, State: Initialized, Role: FOLLOWER
I20260619 14:56:52.403189 16583 consensus_queue.cc:260] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 } } peers { permanent_uuid: "3fc52e9e52d741e0a180eb7d80089255" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 34709 } } peers { permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 } }
I20260619 14:56:52.403825 16585 consensus_queue.cc:260] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 } } peers { permanent_uuid: "3fc52e9e52d741e0a180eb7d80089255" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 34709 } } peers { permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 } }
I20260619 14:56:52.405388 16584 raft_consensus.cc:348] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 } } peers { permanent_uuid: "3fc52e9e52d741e0a180eb7d80089255" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 34709 } } peers { permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 } }
I20260619 14:56:52.405789 16584 raft_consensus.cc:374] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:52.405920 16584 raft_consensus.cc:729] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e151c4f810f4ef48052f8d57aa4cf15, State: Initialized, Role: FOLLOWER
I20260619 14:56:52.406924 16584 consensus_queue.cc:260] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 } } peers { permanent_uuid: "3fc52e9e52d741e0a180eb7d80089255" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 34709 } } peers { permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 } }
I20260619 14:56:52.408098 16433 heartbeater.cc:503] Master 127.11.175.254:42691 was elected leader, sending a full tablet report...
I20260619 14:56:52.408829 16584 ts_tablet_manager.cc:1428] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260619 14:56:52.410558 16565 heartbeater.cc:503] Master 127.11.175.254:42691 was elected leader, sending a full tablet report...
I20260619 14:56:52.411605 16583 ts_tablet_manager.cc:1428] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: Time spent starting tablet: real 0.022s user 0.021s sys 0.001s
I20260619 14:56:52.411734 16585 ts_tablet_manager.cc:1428] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: Time spent starting tablet: real 0.020s user 0.012s sys 0.008s
I20260619 14:56:52.463603 16589 raft_consensus.cc:482] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:52.463972 16589 raft_consensus.cc:504] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 } } peers { permanent_uuid: "3fc52e9e52d741e0a180eb7d80089255" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 34709 } } peers { permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 } }
I20260619 14:56:52.465950 16589 leader_election.cc:288] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aad16fdbf1ce48f0a24a8cb149480425 (127.11.175.193:43103), 8e151c4f810f4ef48052f8d57aa4cf15 (127.11.175.194:45059)
I20260619 14:56:52.473407 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.466248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:39431 (local address 127.11.175.193:43103)
0619 14:56:52.466445 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.466453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.466588 (+ 135us) server_negotiation.cc:407] Connection header received
0619 14:56:52.466853 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.466862 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.466962 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.467070 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:52.467518 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.468555 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.470981 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.472190 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.472360 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.472453 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.473009 (+ 556us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.473144 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 14:56:52.473408 16582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.466400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:58031 (local address 127.11.175.194:45059)
0619 14:56:52.466693 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.466701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.467168 (+ 467us) server_negotiation.cc:407] Connection header received
0619 14:56:52.467281 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.467285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.467339 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.467408 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:56:52.467897 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.468839 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.471169 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.472352 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.472505 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.472552 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.473064 (+ 512us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.473181 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 14:56:52.473614 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.466151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43103 (local address 127.11.175.195:39431)
0619 14:56:52.466444 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:52.466464 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:52.466587 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:52.467115 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:52.467123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:52.467151 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:52.467351 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.467359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.468696 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.468700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.470812 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.470828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.472385 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.472390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.472790 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.472840 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 14:56:52.472974 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 14:56:52.473185 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":34}
I20260619 14:56:52.474048 16592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.466226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:45059 (local address 127.11.175.195:58031)
0619 14:56:52.467048 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:52.467066 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:52.467171 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:52.467454 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:52.467458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:52.467484 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:56:52.467748 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.467755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.468969 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.468974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.471026 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:52.471035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.472573 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.472581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:52.472880 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.472932 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 14:56:52.473031 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 14:56:52.473185 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":60,"thread_start_us":557,"threads_started":1}
I20260619 14:56:52.474828 16255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2c4f8d0d0547ebbc8adae7b2c84750" candidate_uuid: "3fc52e9e52d741e0a180eb7d80089255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aad16fdbf1ce48f0a24a8cb149480425" is_pre_election: true
I20260619 14:56:52.475117 16388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2c4f8d0d0547ebbc8adae7b2c84750" candidate_uuid: "3fc52e9e52d741e0a180eb7d80089255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" is_pre_election: true
I20260619 14:56:52.475332 16255 raft_consensus.cc:2454] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc52e9e52d741e0a180eb7d80089255 in term 0.
I20260619 14:56:52.475699 16388 raft_consensus.cc:2454] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc52e9e52d741e0a180eb7d80089255 in term 0.
I20260619 14:56:52.476085 16454 leader_election.cc:302] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc52e9e52d741e0a180eb7d80089255, aad16fdbf1ce48f0a24a8cb149480425; no voters:
I20260619 14:56:52.476519 16589 raft_consensus.cc:2781] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:52.476696 16589 raft_consensus.cc:482] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:52.476850 16589 raft_consensus.cc:3037] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:52.479535 16589 raft_consensus.cc:504] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 } } peers { permanent_uuid: "3fc52e9e52d741e0a180eb7d80089255" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 34709 } } peers { permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 } }
I20260619 14:56:52.480294 16589 leader_election.cc:288] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [CANDIDATE]: Term 1 election: Requested vote from peers aad16fdbf1ce48f0a24a8cb149480425 (127.11.175.193:43103), 8e151c4f810f4ef48052f8d57aa4cf15 (127.11.175.194:45059)
I20260619 14:56:52.480785 16255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2c4f8d0d0547ebbc8adae7b2c84750" candidate_uuid: "3fc52e9e52d741e0a180eb7d80089255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aad16fdbf1ce48f0a24a8cb149480425"
I20260619 14:56:52.480851 16388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2c4f8d0d0547ebbc8adae7b2c84750" candidate_uuid: "3fc52e9e52d741e0a180eb7d80089255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e151c4f810f4ef48052f8d57aa4cf15"
I20260619 14:56:52.480993 16255 raft_consensus.cc:3037] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:52.481117 16388 raft_consensus.cc:3037] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:52.483317 16255 raft_consensus.cc:2454] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc52e9e52d741e0a180eb7d80089255 in term 1.
I20260619 14:56:52.483711 16454 leader_election.cc:302] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc52e9e52d741e0a180eb7d80089255, aad16fdbf1ce48f0a24a8cb149480425; no voters:
I20260619 14:56:52.483999 16589 raft_consensus.cc:2781] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:52.484004 16388 raft_consensus.cc:2454] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc52e9e52d741e0a180eb7d80089255 in term 1.
I20260619 14:56:52.484515 16589 raft_consensus.cc:686] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 1 LEADER]: Becoming Leader. State: Replica: 3fc52e9e52d741e0a180eb7d80089255, State: Running, Role: LEADER
I20260619 14:56:52.485028 16589 consensus_queue.cc:237] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 } } peers { permanent_uuid: "3fc52e9e52d741e0a180eb7d80089255" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 34709 } } peers { permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 } }
I20260619 14:56:52.489526 16112 catalog_manager.cc:5495] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fc52e9e52d741e0a180eb7d80089255 (127.11.175.195). New cstate: current_term: 1 leader_uuid: "3fc52e9e52d741e0a180eb7d80089255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fc52e9e52d741e0a180eb7d80089255" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 34709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 } health_report { overall_health: UNKNOWN } } }
W20260619 14:56:52.503468 16301 tablet.cc:2369] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 14:56:52.505987 16434 tablet.cc:2369] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:56:52.507952 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.502533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.11.175.195:34709)
0619 14:56:52.502752 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.502756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.502888 (+ 132us) server_negotiation.cc:407] Connection header received
0619 14:56:52.503072 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.503076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.503162 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.503287 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:56:52.503760 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.504463 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.505749 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.506216 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.506309 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.506893 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:56:52.507447 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:56:52.507555 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.507676 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.507743 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260619 14:56:52.511549 16566 tablet.cc:2369] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:56:52.516924 16388 raft_consensus.cc:1261] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15 [term 1 FOLLOWER]: Refusing update from remote peer 3fc52e9e52d741e0a180eb7d80089255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:52.516896 16255 raft_consensus.cc:1261] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425 [term 1 FOLLOWER]: Refusing update from remote peer 3fc52e9e52d741e0a180eb7d80089255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:52.517910 16589 consensus_queue.cc:1035] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aad16fdbf1ce48f0a24a8cb149480425" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:52.518211 16593 consensus_queue.cc:1035] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e151c4f810f4ef48052f8d57aa4cf15" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:52.542934 16112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:52.543197 16112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:52.546403 16112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f0f5ee1fd4c4b34b622cee13b4860a0: Sending DeleteTablet for 3 replicas of tablet dd2c4f8d0d0547ebbc8adae7b2c84750
I20260619 14:56:52.547536 16500 tablet_service.cc:1514] Processing DeleteTablet for tablet dd2c4f8d0d0547ebbc8adae7b2c84750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:52 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:49276
I20260619 14:56:52.547657 16368 tablet_service.cc:1514] Processing DeleteTablet for tablet dd2c4f8d0d0547ebbc8adae7b2c84750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:52 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:43716
I20260619 14:56:52.548332 16235 tablet_service.cc:1514] Processing DeleteTablet for tablet dd2c4f8d0d0547ebbc8adae7b2c84750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:52 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:47782
I20260619 14:56:52.549243 16607 tablet_replica.cc:330] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: stopping tablet replica
I20260619 14:56:52.549664 16607 raft_consensus.cc:2229] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:52.549685 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 16173
I20260619 14:56:52.550148 16607 raft_consensus.cc:2258] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:52.552372 16607 ts_tablet_manager.cc:1905] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:52.562664 16607 ts_tablet_manager.cc:1918] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:56:52.562870 16607 log.cc:1196] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/dd2c4f8d0d0547ebbc8adae7b2c84750
I20260619 14:56:52.563411 16607 ts_tablet_manager.cc:1939] T dd2c4f8d0d0547ebbc8adae7b2c84750 P aad16fdbf1ce48f0a24a8cb149480425: Deleting consensus metadata
I20260619 14:56:52.565356 16605 tablet_replica.cc:330] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: stopping tablet replica
I20260619 14:56:52.566171 16605 raft_consensus.cc:2229] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:56:52.567155 16605 raft_consensus.cc:2258] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:52.570020 16605 ts_tablet_manager.cc:1905] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 14:56:52 dist-test-slave-76rk krb5kdc[16068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:56:52.573717 16606 tablet_replica.cc:330] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: stopping tablet replica
I20260619 14:56:52.574261 16606 raft_consensus.cc:2229] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:52.574380 16097 catalog_manager.cc:4841] TS aad16fdbf1ce48f0a24a8cb149480425 (127.11.175.193:43103): tablet dd2c4f8d0d0547ebbc8adae7b2c84750 (table test-table [id=41bacc35796e43ef8a31fee502ace5cd]) successfully deleted
I20260619 14:56:52.574985 16606 raft_consensus.cc:2258] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:52.577776 16606 ts_tablet_manager.cc:1905] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:52.579586 16605 ts_tablet_manager.cc:1918] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:56:52.579770 16605 log.cc:1196] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/dd2c4f8d0d0547ebbc8adae7b2c84750
I20260619 14:56:52.580302 16605 ts_tablet_manager.cc:1939] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 3fc52e9e52d741e0a180eb7d80089255: Deleting consensus metadata
I20260619 14:56:52.582062 16099 catalog_manager.cc:4841] TS 3fc52e9e52d741e0a180eb7d80089255 (127.11.175.195:34709): tablet dd2c4f8d0d0547ebbc8adae7b2c84750 (table test-table [id=41bacc35796e43ef8a31fee502ace5cd]) successfully deleted
I20260619 14:56:52.582741 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.552059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.11.175.193:43103)
0619 14:56:52.552278 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.552287 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.555107 (+ 2820us) server_negotiation.cc:407] Connection header received
0619 14:56:52.555376 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.555381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.555488 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.555565 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:52.564977 (+ 9412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.566112 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.568121 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.568754 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.568858 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.574938 (+ 6080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:52.574953 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:52.574974 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:52.575032 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:52.579137 (+ 4105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:52.580038 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:52.580048 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:52.580054 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:52.580229 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:52.580705 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:52.580710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:52.580713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:52.581189 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:52.581495 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.582097 (+ 602us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.582335 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 14:56:52.584074 16295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:52.585640 16606 ts_tablet_manager.cc:1918] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:56:52.585914 16606 log.cc:1196] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/dd2c4f8d0d0547ebbc8adae7b2c84750
I20260619 14:56:52.586369 16606 ts_tablet_manager.cc:1939] T dd2c4f8d0d0547ebbc8adae7b2c84750 P 8e151c4f810f4ef48052f8d57aa4cf15: Deleting consensus metadata
I20260619 14:56:52.588861 16096 catalog_manager.cc:4841] TS 8e151c4f810f4ef48052f8d57aa4cf15 (127.11.175.194:45059): tablet dd2c4f8d0d0547ebbc8adae7b2c84750 (table test-table [id=41bacc35796e43ef8a31fee502ace5cd]) successfully deleted
I20260619 14:56:52.677474 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 16173
I20260619 14:56:52.695712 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 16306
Jun 19 14:56:52 dist-test-slave-76rk krb5kdc[16068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:56:52.713291 16582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.696602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43720 (local address 127.11.175.194:45059)
0619 14:56:52.696804 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.696810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.696833 (+ 23us) server_negotiation.cc:407] Connection header received
0619 14:56:52.696978 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.696984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.697071 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.697182 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:52.698433 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.699727 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.701219 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.701850 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.701979 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.706454 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:52.706473 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:52.706495 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:52.706543 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:52.710318 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:52.711154 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:52.711164 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:52.711168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:52.711320 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:52.711673 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:52.711678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:52.711681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:52.712155 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:52.712464 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.712678 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.712920 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 14:56:52.714085 16428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:52.803918 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 16306
I20260619 14:56:52.821813 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 16438
Jun 19 14:56:52 dist-test-slave-76rk krb5kdc[16068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:56:52.840620 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:52.822661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49286 (local address 127.11.175.195:34709)
0619 14:56:52.822878 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:52.822885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:52.822908 (+ 23us) server_negotiation.cc:407] Connection header received
0619 14:56:52.823075 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:52.823081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:52.823169 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:52.823285 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:52.824603 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.825834 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.827756 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:52.828478 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:52.828625 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:52.833987 (+ 5362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:52.834002 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:52.834021 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:56:52.834079 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:52.837456 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:52.838209 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:52.838216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:52.838219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:52.838382 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:56:52.838978 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:56:52.838983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:56:52.838985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:56:52.839361 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:52.839706 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:52.839996 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 14:56:52.840221 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 14:56:52.841348 16560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:52.967223 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 16438
I20260619 14:56:52.986799 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 16081
I20260619 14:56:52.987995 16142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:53.098762 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 16081
2026-06-19T14:56:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4206 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T14:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:53Z Disabled control of system clock
I20260619 14:56:53.169260 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:35237
--webserver_interface=127.11.175.254
--webserver_port=0
--builtin_ntp_servers=127.11.175.212:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:35237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 14:56:53.503167 16616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:53.503468 16616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:53.503635 16616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:53.511116 16616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:53.511230 16616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:53.511323 16616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:53.511426 16616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:53.511515 16616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:56:53.521127 16616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:35237
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:35237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:53.522454 16616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:53.524251 16616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:53.536382 16624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:53.536528 16621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:53.536509 16622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:53.537259 16616 server_base.cc:1034] running on GCE node
I20260619 14:56:53.538118 16616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:53.539678 16616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:53.540946 16616 hybrid_clock.cc:648] HybridClock initialized: now 1781881013540873 us; error 37 us; skew 500 ppm
I20260619 14:56:53.541318 16616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:56:53.543749 16616 webserver.cc:460] Webserver started at http://127.11.175.254:35633/ using document root <none> and password file <none>
I20260619 14:56:53.544473 16616 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:53.544607 16616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:53.544948 16616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:53.547884 16616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "cb46ec80fa71410abb39262bfe903132"
format_stamp: "Formatted at 2026-06-19 14:56:53 on dist-test-slave-76rk"
server_key: "8cc358c09b58c4b92d02a89516a3af3c"
server_key_iv: "5d7c441f8d6b74a0c0c4514d8a92e1c8"
server_key_version: "encryptionkey@0"
I20260619 14:56:53.548719 16616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "cb46ec80fa71410abb39262bfe903132"
format_stamp: "Formatted at 2026-06-19 14:56:53 on dist-test-slave-76rk"
server_key: "8cc358c09b58c4b92d02a89516a3af3c"
server_key_iv: "5d7c441f8d6b74a0c0c4514d8a92e1c8"
server_key_version: "encryptionkey@0"
I20260619 14:56:53.553947 16616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 14:56:53.557389 16630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:53.558797 16616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:56:53.559012 16616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "cb46ec80fa71410abb39262bfe903132"
format_stamp: "Formatted at 2026-06-19 14:56:53 on dist-test-slave-76rk"
server_key: "8cc358c09b58c4b92d02a89516a3af3c"
server_key_iv: "5d7c441f8d6b74a0c0c4514d8a92e1c8"
server_key_version: "encryptionkey@0"
I20260619 14:56:53.559248 16616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:53.565028 16616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:53.565822 16616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:53.566136 16616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:53.584794 16616 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:35237
I20260619 14:56:53.584885 16681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:35237 every 8 connection(s)
I20260619 14:56:53.586223 16616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:53.590054 16682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:53.591605 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 16616
I20260619 14:56:53.591917 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:53.592609 11967 external_mini_cluster.cc:1428] Setting key a6e972eab172ee93072882bf3c898516
I20260619 14:56:53.603155 16682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132: Bootstrap starting.
I20260619 14:56:53.605863 16685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:53.594803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45328 (local address 127.11.175.254:35237)
0619 14:56:53.596109 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:53.596131 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:53.596188 (+ 57us) server_negotiation.cc:407] Connection header received
0619 14:56:53.598535 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:53.598563 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:53.599125 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:53.599580 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:53.603732 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:53.603741 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:53.603758 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:53.603777 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:53.603861 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:53.603884 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:53.604101 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:53.604238 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 14:56:53.604533 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":344,"threads_started":1}
I20260619 14:56:53.607008 16682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:53.608100 16682 log.cc:824] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:53.611014 16682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132: No bootstrap required, opened a new log
I20260619 14:56:53.618553 16682 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb46ec80fa71410abb39262bfe903132" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35237 } }
I20260619 14:56:53.619004 16682 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:53.619128 16682 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb46ec80fa71410abb39262bfe903132, State: Initialized, Role: FOLLOWER
I20260619 14:56:53.619767 16682 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb46ec80fa71410abb39262bfe903132" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35237 } }
I20260619 14:56:53.620002 16682 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:53.620131 16682 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:53.620304 16682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:53.622964 16682 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb46ec80fa71410abb39262bfe903132" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35237 } }
I20260619 14:56:53.623518 16682 leader_election.cc:302] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb46ec80fa71410abb39262bfe903132; no voters:
I20260619 14:56:53.624159 16682 leader_election.cc:288] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:53.624310 16687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:53.625463 16687 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [term 1 LEADER]: Becoming Leader. State: Replica: cb46ec80fa71410abb39262bfe903132, State: Running, Role: LEADER
I20260619 14:56:53.626135 16687 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb46ec80fa71410abb39262bfe903132" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35237 } }
I20260619 14:56:53.626302 16682 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:53.629685 16689 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb46ec80fa71410abb39262bfe903132. Latest consensus state: current_term: 1 leader_uuid: "cb46ec80fa71410abb39262bfe903132" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb46ec80fa71410abb39262bfe903132" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35237 } } }
I20260619 14:56:53.629709 16688 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb46ec80fa71410abb39262bfe903132" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb46ec80fa71410abb39262bfe903132" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35237 } } }
I20260619 14:56:53.630123 16689 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:53.630127 16688 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:53.631886 16697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:53.636505 16697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:53.644449 16697 catalog_manager.cc:1347] Generated new cluster ID: 2a284234c7cb445b9edf73bdc1961294
I20260619 14:56:53.644564 16697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:53.695065 16697 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:53.695533 16697 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:53.709595 16697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132: Generated new TSK 0
I20260619 14:56:53.710503 16697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 14:56:53.721833 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35237
--builtin_ntp_servers=127.11.175.212:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 14:56:54.056422 16706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:54.056782 16706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:54.056962 16706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:54.064357 16706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:54.064456 16706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:54.064566 16706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:56:54.074448 16706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:54.075827 16706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:54.077597 16706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:54.091619 16712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:54.091832 16711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:54.091632 16714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:54.092413 16706 server_base.cc:1034] running on GCE node
I20260619 14:56:54.093082 16706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:54.093999 16706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:54.095194 16706 hybrid_clock.cc:648] HybridClock initialized: now 1781881014095124 us; error 59 us; skew 500 ppm
I20260619 14:56:54.095510 16706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:56:54.097844 16706 webserver.cc:460] Webserver started at http://127.11.175.193:41959/ using document root <none> and password file <none>
I20260619 14:56:54.098495 16706 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:54.098582 16706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:54.098940 16706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:54.101728 16706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "be4bccb6143c48b99ef971690b700c56"
format_stamp: "Formatted at 2026-06-19 14:56:54 on dist-test-slave-76rk"
server_key: "d7475175fcd139b0ee792cfa717c8ee6"
server_key_iv: "d9e7d87aa7f26819ba54a5e1757582f7"
server_key_version: "encryptionkey@0"
I20260619 14:56:54.102465 16706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "be4bccb6143c48b99ef971690b700c56"
format_stamp: "Formatted at 2026-06-19 14:56:54 on dist-test-slave-76rk"
server_key: "d7475175fcd139b0ee792cfa717c8ee6"
server_key_iv: "d9e7d87aa7f26819ba54a5e1757582f7"
server_key_version: "encryptionkey@0"
I20260619 14:56:54.107743 16706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 14:56:54.111166 16720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:54.112618 16706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 14:56:54.112761 16706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "be4bccb6143c48b99ef971690b700c56"
format_stamp: "Formatted at 2026-06-19 14:56:54 on dist-test-slave-76rk"
server_key: "d7475175fcd139b0ee792cfa717c8ee6"
server_key_iv: "d9e7d87aa7f26819ba54a5e1757582f7"
server_key_version: "encryptionkey@0"
I20260619 14:56:54.112960 16706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:54.118625 16706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:54.119460 16706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:54.119684 16706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:54.120599 16706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:54.121976 16706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:54.122061 16706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:54.122140 16706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:54.122195 16706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:54.157011 16706 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:39559
I20260619 14:56:54.157100 16832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:39559 every 8 connection(s)
I20260619 14:56:54.158370 16706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:56:54.163252 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 16706
I20260619 14:56:54.163444 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:56:54.164232 11967 external_mini_cluster.cc:1428] Setting key fd6d7b5fd6fb139ac45306d05b56a4cc
I20260619 14:56:54.167261 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35237
--builtin_ntp_servers=127.11.175.212:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 14:56:54.167716 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:54.161110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:42941 (local address 127.11.175.254:35237)
0619 14:56:54.161541 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:54.161546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:54.162797 (+ 1251us) server_negotiation.cc:407] Connection header received
0619 14:56:54.165253 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:54.165260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:54.165327 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:54.165426 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:54.166935 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:54.166946 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:54.166951 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:54.166966 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:54.167007 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:54.167019 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:54.167149 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:54.167358 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 14:56:54.167426 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":233,"threads_started":1}
I20260619 14:56:54.168680 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:54.161683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35237 (local address 127.11.175.193:42941)
0619 14:56:54.162597 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:54.162683 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:54.164940 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:54.165699 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:54.165710 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:54.165831 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:54.166563 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:56:54.166565 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:54.166610 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:56:54.166760 (+ 150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:54.167118 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:54.167131 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:54.167139 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 14:56:54.167452 (+ 313us) client_negotiation.cc:240] Negotiation successful
0619 14:56:54.167720 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":272,"threads_started":1}
I20260619 14:56:54.170288 16833 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35237
I20260619 14:56:54.170629 16833 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:54.171273 16833 heartbeater.cc:511] Master 127.11.175.254:35237 requested a full tablet report, sending...
I20260619 14:56:54.173384 16647 ts_manager.cc:194] Registered new tserver with Master: be4bccb6143c48b99ef971690b700c56 (127.11.175.193:39559)
W20260619 14:56:54.497659 16837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:54.497951 16837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:54.498122 16837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:54.505546 16837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:54.505663 16837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:54.505879 16837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:56:54.515492 16837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:54.516974 16837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:54.518906 16837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:54.532773 16843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:54.533083 16842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:54.532773 16845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:54.533986 16837 server_base.cc:1034] running on GCE node
I20260619 14:56:54.534554 16837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:54.535583 16837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:54.536775 16837 hybrid_clock.cc:648] HybridClock initialized: now 1781881014536726 us; error 41 us; skew 500 ppm
I20260619 14:56:54.537106 16837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:56:54.539382 16837 webserver.cc:460] Webserver started at http://127.11.175.194:41447/ using document root <none> and password file <none>
I20260619 14:56:54.540110 16837 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:54.540198 16837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:54.540566 16837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:54.543615 16837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "989221cecfb74e14b1defa719da901e1"
format_stamp: "Formatted at 2026-06-19 14:56:54 on dist-test-slave-76rk"
server_key: "445df50332b0c8e920f031b8bb5015e5"
server_key_iv: "1288319c5ed8f86b064b8e306c83dbce"
server_key_version: "encryptionkey@0"
I20260619 14:56:54.544402 16837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "989221cecfb74e14b1defa719da901e1"
format_stamp: "Formatted at 2026-06-19 14:56:54 on dist-test-slave-76rk"
server_key: "445df50332b0c8e920f031b8bb5015e5"
server_key_iv: "1288319c5ed8f86b064b8e306c83dbce"
server_key_version: "encryptionkey@0"
I20260619 14:56:54.549448 16837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 14:56:54.552831 16851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:54.554148 16837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 14:56:54.554345 16837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "989221cecfb74e14b1defa719da901e1"
format_stamp: "Formatted at 2026-06-19 14:56:54 on dist-test-slave-76rk"
server_key: "445df50332b0c8e920f031b8bb5015e5"
server_key_iv: "1288319c5ed8f86b064b8e306c83dbce"
server_key_version: "encryptionkey@0"
I20260619 14:56:54.554579 16837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:54.560408 16837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:54.561203 16837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:54.561473 16837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:54.562358 16837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:54.564073 16837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:54.564194 16837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:54.564325 16837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:54.564424 16837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:54.597702 16837 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:41473
I20260619 14:56:54.597865 16963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:41473 every 8 connection(s)
I20260619 14:56:54.599232 16837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:56:54.607637 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:54.602011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:48467 (local address 127.11.175.254:35237)
0619 14:56:54.602222 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:54.602228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:54.603881 (+ 1653us) server_negotiation.cc:407] Connection header received
0619 14:56:54.605519 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:54.605525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:54.605604 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:54.605715 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:54.607039 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:54.607043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:54.607046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:54.607061 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:54.607093 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:54.607104 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:54.607195 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:54.607368 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 14:56:54.607417 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 14:56:54.608525 16966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:54.602587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35237 (local address 127.11.175.194:48467)
0619 14:56:54.603704 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:54.603746 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:54.605258 (+ 1512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:54.605920 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:54.605928 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:54.606018 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:54.606747 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:56:54.606749 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:54.606792 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:56:54.606878 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:54.607188 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:54.607195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:54.607200 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 14:56:54.607421 (+ 221us) client_negotiation.cc:240] Negotiation successful
0619 14:56:54.607617 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":331,"threads_started":1}
I20260619 14:56:54.609068 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 16837
I20260619 14:56:54.609233 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:56:54.609817 11967 external_mini_cluster.cc:1428] Setting key 6e77df29189ae2c30ada1b92917a3fcf
I20260619 14:56:54.609911 16964 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35237
I20260619 14:56:54.610219 16964 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:54.610929 16964 heartbeater.cc:511] Master 127.11.175.254:35237 requested a full tablet report, sending...
I20260619 14:56:54.612386 16647 ts_manager.cc:194] Registered new tserver with Master: 989221cecfb74e14b1defa719da901e1 (127.11.175.194:41473)
I20260619 14:56:54.612830 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35237
--builtin_ntp_servers=127.11.175.212:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 14:56:54.946161 16967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:54.946453 16967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:54.946549 16967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:54.953954 16967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:54.954057 16967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:54.954165 16967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:56:54.963521 16967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:54.964808 16967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:54.966610 16967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:54.980020 16972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:54.979983 16975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:54.980068 16973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:54.992439 16967 server_base.cc:1034] running on GCE node
I20260619 14:56:54.993099 16967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:54.994112 16967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:54.995373 16967 hybrid_clock.cc:648] HybridClock initialized: now 1781881014995319 us; error 45 us; skew 500 ppm
I20260619 14:56:54.995769 16967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:56:54.998255 16967 webserver.cc:460] Webserver started at http://127.11.175.195:42013/ using document root <none> and password file <none>
I20260619 14:56:54.999092 16967 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:54.999229 16967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:54.999579 16967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:55.002604 16967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "f03408b2737d44c18d976c806e69f8b2"
format_stamp: "Formatted at 2026-06-19 14:56:54 on dist-test-slave-76rk"
server_key: "8efc40c979542ab4f36ca05484ce4147"
server_key_iv: "e96cb26d80c1a044a050727a1aee03f9"
server_key_version: "encryptionkey@0"
I20260619 14:56:55.003479 16967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "f03408b2737d44c18d976c806e69f8b2"
format_stamp: "Formatted at 2026-06-19 14:56:54 on dist-test-slave-76rk"
server_key: "8efc40c979542ab4f36ca05484ce4147"
server_key_iv: "e96cb26d80c1a044a050727a1aee03f9"
server_key_version: "encryptionkey@0"
I20260619 14:56:55.008850 16967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:56:55.012459 16981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:55.013967 16967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 14:56:55.014155 16967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "f03408b2737d44c18d976c806e69f8b2"
format_stamp: "Formatted at 2026-06-19 14:56:54 on dist-test-slave-76rk"
server_key: "8efc40c979542ab4f36ca05484ce4147"
server_key_iv: "e96cb26d80c1a044a050727a1aee03f9"
server_key_version: "encryptionkey@0"
I20260619 14:56:55.014396 16967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:55.020229 16967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:55.021044 16967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:55.021319 16967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:55.022233 16967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:56:55.023751 16967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:56:55.023839 16967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:55.023922 16967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:56:55.023976 16967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:55.057211 16967 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:38697
I20260619 14:56:55.057262 17093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:38697 every 8 connection(s)
I20260619 14:56:55.058609 16967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:56:55.063519 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 16967
I20260619 14:56:55.063685 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:56:55.064162 11967 external_mini_cluster.cc:1428] Setting key a4d66ae3537e009ed9468a7eaee46b6d
I20260619 14:56:55.067252 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.061157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:58767 (local address 127.11.175.254:35237)
0619 14:56:55.061326 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.061331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.062786 (+ 1455us) server_negotiation.cc:407] Connection header received
0619 14:56:55.064828 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.064834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.064919 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.065031 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.066448 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.066454 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.066459 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.066484 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.066516 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.066528 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.066672 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.066874 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.066964 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 14:56:55.068215 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.061620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35237 (local address 127.11.175.195:58767)
0619 14:56:55.062580 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.062615 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:55.064546 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:55.065264 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:55.065272 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:55.065405 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:55.066151 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:56:55.066154 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:55.066190 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:56:55.066275 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:55.066626 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:55.066663 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:55.066671 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 14:56:55.066964 (+ 293us) client_negotiation.cc:240] Negotiation successful
0619 14:56:55.067224 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":268,"threads_started":1}
I20260619 14:56:55.069478 17094 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35237
I20260619 14:56:55.069773 17094 heartbeater.cc:464] Registering TS with master...
I20260619 14:56:55.070363 17094 heartbeater.cc:511] Master 127.11.175.254:35237 requested a full tablet report, sending...
I20260619 14:56:55.071902 16647 ts_manager.cc:194] Registered new tserver with Master: f03408b2737d44c18d976c806e69f8b2 (127.11.175.195:38697)
I20260619 14:56:55.080104 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 14:56:55.087107 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.084600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45336 (local address 127.11.175.254:35237)
0619 14:56:55.084768 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.084772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.085325 (+ 553us) server_negotiation.cc:407] Connection header received
0619 14:56:55.085465 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.085469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.085521 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.085604 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.086425 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.086430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.086433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.086447 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.086478 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.086489 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.086587 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.086741 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.086789 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 14:56:55.090972 16647 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:56:55.098259 16647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:56:55.126566 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.110765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38697 (local address 127.0.0.1:46890)
0619 14:56:55.121588 (+ 10823us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.121629 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:55.121806 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:55.122656 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:55.122668 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:55.122800 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:55.123794 (+ 994us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:56:55.123801 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:55.123844 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:56:55.123943 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:55.125573 (+ 1630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:55.125590 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:55.125602 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 14:56:55.125814 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 14:56:55.125924 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10560,"thread_start_us":10465,"threads_started":1}
I20260619 14:56:55.128333 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.111006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46890 (local address 127.11.175.195:38697)
0619 14:56:55.121192 (+ 10186us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.121205 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.121944 (+ 739us) server_negotiation.cc:407] Connection header received
0619 14:56:55.122034 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.122041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.122378 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.123378 (+ 1000us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.125260 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.125272 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.125293 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.125315 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.125381 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.125406 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.127585 (+ 2179us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.127670 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.127800 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9983,"thread_start_us":9793,"threads_started":1}
I20260619 14:56:55.132082 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.122384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48354 (local address 127.11.175.194:41473)
0619 14:56:55.129797 (+ 7413us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.129806 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.129837 (+ 31us) server_negotiation.cc:407] Connection header received
0619 14:56:55.129916 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.129923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.130152 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.130772 (+ 620us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.131383 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.131388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.131399 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.131410 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.131457 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.131474 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.131631 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.131700 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.131799 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7207,"thread_start_us":7471,"threads_started":1}
I20260619 14:56:55.132059 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.126572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:41473 (local address 127.0.0.1:48354)
0619 14:56:55.127194 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.127217 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:55.127318 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:55.130378 (+ 3060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:55.130385 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:55.130453 (+ 68us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:55.131068 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:56:55.131073 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:55.131094 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:56:55.131143 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:55.131589 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:55.131594 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:55.131598 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 14:56:55.131691 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 14:56:55.131744 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":2542,"threads_started":1}
I20260619 14:56:55.132416 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.110781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39559 (local address 127.0.0.1:35956)
0619 14:56:55.126753 (+ 15972us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.126779 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:55.126930 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:55.128318 (+ 1388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:55.128327 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:55.128363 (+ 36us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:55.129359 (+ 996us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:56:55.129362 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:55.129381 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:56:55.129426 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:55.131677 (+ 2251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:55.131681 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:55.131686 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 14:56:55.131812 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 14:56:55.131872 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15555,"thread_start_us":15620,"threads_started":1}
I20260619 14:56:55.133140 17106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.110781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.11.175.193:39559)
0619 14:56:55.126210 (+ 15429us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.126221 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.127089 (+ 868us) server_negotiation.cc:407] Connection header received
0619 14:56:55.127187 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.127200 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.128077 (+ 877us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.129609 (+ 1532us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.131418 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.131426 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.131439 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.131465 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.131518 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.131538 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.132634 (+ 1096us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.132711 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.132836 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15223,"thread_start_us":15447,"threads_started":1}
I20260619 14:56:55.134800 17029 tablet_service.cc:1467] Processing CreateTablet for tablet a29371b5f02e450e919166fb79d1c473 (DEFAULT_TABLE table=test-table [id=8f775491b32a4aeaac42e94b78abd597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:55.135950 16899 tablet_service.cc:1467] Processing CreateTablet for tablet a29371b5f02e450e919166fb79d1c473 (DEFAULT_TABLE table=test-table [id=8f775491b32a4aeaac42e94b78abd597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:55.136412 17029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a29371b5f02e450e919166fb79d1c473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:55.136636 16768 tablet_service.cc:1467] Processing CreateTablet for tablet a29371b5f02e450e919166fb79d1c473 (DEFAULT_TABLE table=test-table [id=8f775491b32a4aeaac42e94b78abd597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:56:55.137184 16899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a29371b5f02e450e919166fb79d1c473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:55.137892 16768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a29371b5f02e450e919166fb79d1c473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:55.148772 17112 tablet_bootstrap.cc:492] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: Bootstrap starting.
I20260619 14:56:55.151219 17113 tablet_bootstrap.cc:492] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: Bootstrap starting.
I20260619 14:56:55.152529 17114 tablet_bootstrap.cc:492] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56: Bootstrap starting.
I20260619 14:56:55.153019 17112 tablet_bootstrap.cc:654] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:55.154382 17112 log.cc:824] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:55.155652 17113 tablet_bootstrap.cc:654] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:55.155723 17114 tablet_bootstrap.cc:654] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:55.156790 17113 log.cc:824] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:55.156956 17114 log.cc:824] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:55.158054 17112 tablet_bootstrap.cc:492] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: No bootstrap required, opened a new log
I20260619 14:56:55.158490 17112 ts_tablet_manager.cc:1397] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260619 14:56:55.170109 17114 tablet_bootstrap.cc:492] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56: No bootstrap required, opened a new log
I20260619 14:56:55.170496 17114 ts_tablet_manager.cc:1397] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.009s
I20260619 14:56:55.170995 17113 tablet_bootstrap.cc:492] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: No bootstrap required, opened a new log
I20260619 14:56:55.171294 17113 ts_tablet_manager.cc:1397] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.012s
I20260619 14:56:55.172484 17112 raft_consensus.cc:348] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 } } peers { permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 } } peers { permanent_uuid: "989221cecfb74e14b1defa719da901e1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41473 } }
I20260619 14:56:55.173038 17112 raft_consensus.cc:374] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:55.173182 17112 raft_consensus.cc:729] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989221cecfb74e14b1defa719da901e1, State: Initialized, Role: FOLLOWER
I20260619 14:56:55.173897 17112 consensus_queue.cc:260] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 } } peers { permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 } } peers { permanent_uuid: "989221cecfb74e14b1defa719da901e1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41473 } }
I20260619 14:56:55.175323 16964 heartbeater.cc:503] Master 127.11.175.254:35237 was elected leader, sending a full tablet report...
I20260619 14:56:55.175956 17112 ts_tablet_manager.cc:1428] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260619 14:56:55.176184 16833 heartbeater.cc:503] Master 127.11.175.254:35237 was elected leader, sending a full tablet report...
I20260619 14:56:55.178457 17118 raft_consensus.cc:482] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:56:55.178767 17118 raft_consensus.cc:504] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 } } peers { permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 } } peers { permanent_uuid: "989221cecfb74e14b1defa719da901e1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41473 } }
I20260619 14:56:55.182758 17113 raft_consensus.cc:348] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 } } peers { permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 } } peers { permanent_uuid: "989221cecfb74e14b1defa719da901e1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41473 } }
I20260619 14:56:55.183233 17113 raft_consensus.cc:374] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:55.182231 17114 raft_consensus.cc:348] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 } } peers { permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 } } peers { permanent_uuid: "989221cecfb74e14b1defa719da901e1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41473 } }
I20260619 14:56:55.183404 17113 raft_consensus.cc:729] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f03408b2737d44c18d976c806e69f8b2, State: Initialized, Role: FOLLOWER
I20260619 14:56:55.183620 17114 raft_consensus.cc:374] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:55.183776 17114 raft_consensus.cc:729] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be4bccb6143c48b99ef971690b700c56, State: Initialized, Role: FOLLOWER
I20260619 14:56:55.184309 17113 consensus_queue.cc:260] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 } } peers { permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 } } peers { permanent_uuid: "989221cecfb74e14b1defa719da901e1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41473 } }
I20260619 14:56:55.184373 17114 consensus_queue.cc:260] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 } } peers { permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 } } peers { permanent_uuid: "989221cecfb74e14b1defa719da901e1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41473 } }
I20260619 14:56:55.186977 17118 leader_election.cc:288] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be4bccb6143c48b99ef971690b700c56 (127.11.175.193:39559), f03408b2737d44c18d976c806e69f8b2 (127.11.175.195:38697)
I20260619 14:56:55.188575 17094 heartbeater.cc:503] Master 127.11.175.254:35237 was elected leader, sending a full tablet report...
I20260619 14:56:55.189626 17113 ts_tablet_manager.cc:1428] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: Time spent starting tablet: real 0.018s user 0.009s sys 0.006s
I20260619 14:56:55.191358 17114 ts_tablet_manager.cc:1428] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56: Time spent starting tablet: real 0.021s user 0.017s sys 0.000s
I20260619 14:56:55.197566 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.181034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:54103 (local address 127.11.175.195:38697)
0619 14:56:55.181207 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.181214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.195181 (+ 13967us) server_negotiation.cc:407] Connection header received
0619 14:56:55.195265 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.195274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.195348 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.195459 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.196948 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.196959 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.196963 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.196978 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.197008 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.197020 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.197159 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.197235 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.197300 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 14:56:55.197752 17106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.182047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:54027 (local address 127.11.175.193:39559)
0619 14:56:55.182200 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.182206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.195642 (+ 13436us) server_negotiation.cc:407] Connection header received
0619 14:56:55.195867 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.195872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.195930 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.196034 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.197108 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.197114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.197118 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.197138 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.197177 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.197188 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.197306 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.197401 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.197466 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 14:56:55.197876 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.181124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38697 (local address 127.11.175.194:54103)
0619 14:56:55.194539 (+ 13415us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.194562 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:55.194724 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:55.195926 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:55.195931 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:55.195955 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:55.196710 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:56:55.196713 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:55.196728 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:56:55.196783 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:55.197121 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:55.197126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:55.197130 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 14:56:55.197287 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 14:56:55.197339 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13223,"thread_start_us":14087,"threads_started":1}
I20260619 14:56:55.198004 17119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.181592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39559 (local address 127.11.175.194:54027)
0619 14:56:55.195547 (+ 13955us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.195567 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:55.195726 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:55.196170 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:55.196173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:55.196196 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:56:55.196917 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:56:55.196920 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:56:55.196933 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:56:55.196969 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:56:55.197282 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:56:55.197287 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:56:55.197292 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 14:56:55.197446 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 14:56:55.197499 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13811,"thread_start_us":13236,"threads_started":1}
I20260619 14:56:55.198923 17049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a29371b5f02e450e919166fb79d1c473" candidate_uuid: "989221cecfb74e14b1defa719da901e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f03408b2737d44c18d976c806e69f8b2" is_pre_election: true
I20260619 14:56:55.198933 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a29371b5f02e450e919166fb79d1c473" candidate_uuid: "989221cecfb74e14b1defa719da901e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be4bccb6143c48b99ef971690b700c56" is_pre_election: true
I20260619 14:56:55.199398 17049 raft_consensus.cc:2454] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 989221cecfb74e14b1defa719da901e1 in term 0.
I20260619 14:56:55.199411 16788 raft_consensus.cc:2454] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 989221cecfb74e14b1defa719da901e1 in term 0.
I20260619 14:56:55.200003 16855 leader_election.cc:302] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 989221cecfb74e14b1defa719da901e1, f03408b2737d44c18d976c806e69f8b2; no voters:
I20260619 14:56:55.200397 17118 raft_consensus.cc:2781] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:56:55.200551 17118 raft_consensus.cc:482] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:56:55.200655 17118 raft_consensus.cc:3037] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:55.202874 17118 raft_consensus.cc:504] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 } } peers { permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 } } peers { permanent_uuid: "989221cecfb74e14b1defa719da901e1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41473 } }
I20260619 14:56:55.203516 17118 leader_election.cc:288] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [CANDIDATE]: Term 1 election: Requested vote from peers be4bccb6143c48b99ef971690b700c56 (127.11.175.193:39559), f03408b2737d44c18d976c806e69f8b2 (127.11.175.195:38697)
I20260619 14:56:55.203853 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a29371b5f02e450e919166fb79d1c473" candidate_uuid: "989221cecfb74e14b1defa719da901e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be4bccb6143c48b99ef971690b700c56"
I20260619 14:56:55.203963 17049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a29371b5f02e450e919166fb79d1c473" candidate_uuid: "989221cecfb74e14b1defa719da901e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f03408b2737d44c18d976c806e69f8b2"
I20260619 14:56:55.204063 16788 raft_consensus.cc:3037] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:55.204190 17049 raft_consensus.cc:3037] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:55.206301 16788 raft_consensus.cc:2454] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 989221cecfb74e14b1defa719da901e1 in term 1.
I20260619 14:56:55.206467 17049 raft_consensus.cc:2454] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 989221cecfb74e14b1defa719da901e1 in term 1.
I20260619 14:56:55.206728 16853 leader_election.cc:302] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 989221cecfb74e14b1defa719da901e1, be4bccb6143c48b99ef971690b700c56; no voters:
I20260619 14:56:55.207079 17118 raft_consensus.cc:2781] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:55.207515 17118 raft_consensus.cc:686] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 1 LEADER]: Becoming Leader. State: Replica: 989221cecfb74e14b1defa719da901e1, State: Running, Role: LEADER
I20260619 14:56:55.207984 17118 consensus_queue.cc:237] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 } } peers { permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 } } peers { permanent_uuid: "989221cecfb74e14b1defa719da901e1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41473 } }
I20260619 14:56:55.212889 16647 catalog_manager.cc:5495] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 989221cecfb74e14b1defa719da901e1 (127.11.175.194). New cstate: current_term: 1 leader_uuid: "989221cecfb74e14b1defa719da901e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "989221cecfb74e14b1defa719da901e1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41473 } health_report { overall_health: HEALTHY } } }
I20260619 14:56:55.239776 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.237233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.11.175.194:41473)
0619 14:56:55.237463 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.237471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.237561 (+ 90us) server_negotiation.cc:407] Connection header received
0619 14:56:55.237842 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.237847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.237933 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.238061 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.239025 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.239031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.239044 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.239061 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.239095 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.239109 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.239241 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.239404 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.239466 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260619 14:56:55.248503 16788 raft_consensus.cc:1261] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56 [term 1 FOLLOWER]: Refusing update from remote peer 989221cecfb74e14b1defa719da901e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:55.248513 17049 raft_consensus.cc:1261] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2 [term 1 FOLLOWER]: Refusing update from remote peer 989221cecfb74e14b1defa719da901e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:56:55.249526 17123 consensus_queue.cc:1035] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f03408b2737d44c18d976c806e69f8b2" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:55.249869 17118 consensus_queue.cc:1035] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be4bccb6143c48b99ef971690b700c56" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:56:55.264341 17128 mvcc.cc:205] Tried to move back new op lower bound from 7298584638447632384 to 7298584638296051712. Current Snapshot: MvccSnapshot[applied={T|T < 7298584638447632384}]
I20260619 14:56:55.266943 17126 mvcc.cc:205] Tried to move back new op lower bound from 7298584638447632384 to 7298584638296051712. Current Snapshot: MvccSnapshot[applied={T|T < 7298584638447632384 or (T in {7298584638447632384})}]
I20260619 14:56:55.279278 16647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:55.279577 16647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:56:55.283077 16647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb46ec80fa71410abb39262bfe903132: Sending DeleteTablet for 3 replicas of tablet a29371b5f02e450e919166fb79d1c473
I20260619 14:56:55.284346 16768 tablet_service.cc:1514] Processing DeleteTablet for tablet a29371b5f02e450e919166fb79d1c473 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:55 UTC) from {username='slave'} at 127.0.0.1:35956
I20260619 14:56:55.284680 16899 tablet_service.cc:1514] Processing DeleteTablet for tablet a29371b5f02e450e919166fb79d1c473 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:55 UTC) from {username='slave'} at 127.0.0.1:48354
I20260619 14:56:55.284776 17029 tablet_service.cc:1514] Processing DeleteTablet for tablet a29371b5f02e450e919166fb79d1c473 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:56:55 UTC) from {username='slave'} at 127.0.0.1:46890
I20260619 14:56:55.286759 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 16706
I20260619 14:56:55.314014 17138 tablet_replica.cc:330] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: stopping tablet replica
I20260619 14:56:55.315989 17138 raft_consensus.cc:2229] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:56:55.317946 17106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.288238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.11.175.193:39559)
0619 14:56:55.288483 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.288492 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.312879 (+ 24387us) server_negotiation.cc:407] Connection header received
0619 14:56:55.314394 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.314418 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.314579 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.314748 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.316847 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.316856 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.316864 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.316893 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.316943 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.316965 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.317099 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.317378 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.317498 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 14:56:55.319260 17138 raft_consensus.cc:2258] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:55.320421 16828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:55.323614 17138 ts_tablet_manager.cc:1905] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:55.327471 17137 tablet_replica.cc:330] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: stopping tablet replica
I20260619 14:56:55.329738 17137 raft_consensus.cc:2229] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:56:55.330909 17137 raft_consensus.cc:2258] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:56:55.338524 17137 ts_tablet_manager.cc:1905] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:56:55.340637 17138 ts_tablet_manager.cc:1918] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:56:55.340885 17138 log.cc:1196] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/a29371b5f02e450e919166fb79d1c473
I20260619 14:56:55.341516 17138 ts_tablet_manager.cc:1939] T a29371b5f02e450e919166fb79d1c473 P f03408b2737d44c18d976c806e69f8b2: Deleting consensus metadata
I20260619 14:56:55.343664 16634 catalog_manager.cc:4841] TS f03408b2737d44c18d976c806e69f8b2 (127.11.175.195:38697): tablet a29371b5f02e450e919166fb79d1c473 (table test-table [id=8f775491b32a4aeaac42e94b78abd597]) successfully deleted
I20260619 14:56:55.347987 17137 ts_tablet_manager.cc:1918] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:56:55.348198 17137 log.cc:1196] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/a29371b5f02e450e919166fb79d1c473
I20260619 14:56:55.348865 17137 ts_tablet_manager.cc:1939] T a29371b5f02e450e919166fb79d1c473 P 989221cecfb74e14b1defa719da901e1: Deleting consensus metadata
I20260619 14:56:55.351125 16634 catalog_manager.cc:4841] TS 989221cecfb74e14b1defa719da901e1 (127.11.175.194:41473): tablet a29371b5f02e450e919166fb79d1c473 (table test-table [id=8f775491b32a4aeaac42e94b78abd597]) successfully deleted
I20260619 14:56:55.428716 17136 tablet_replica.cc:330] T a29371b5f02e450e919166fb79d1c473 P be4bccb6143c48b99ef971690b700c56: stopping tablet replica
I20260619 14:56:55.429009 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 16706
I20260619 14:56:55.447652 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 16837
W20260619 14:56:55.447695 16632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.175.193:39559 (error 108)
I20260619 14:56:55.450827 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.450276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39559 (local address 127.0.0.1:35984)
0619 14:56:55.450466 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.450619 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 14:56:55.451061 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.448731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48374 (local address 127.11.175.194:41473)
0619 14:56:55.448909 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.448915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.448936 (+ 21us) server_negotiation.cc:407] Connection header received
0619 14:56:55.449038 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.449044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.449110 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.449223 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.450289 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.450295 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.450301 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.450320 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.450362 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.450378 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.450510 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.450682 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.450744 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260619 14:56:55.451074 16632 catalog_manager.cc:4584] TS be4bccb6143c48b99ef971690b700c56 (127.11.175.193:39559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a29371b5f02e450e919166fb79d1c473: Network error: Client connection negotiation failed: client connection to 127.11.175.193:39559: connect: Connection refused (error 111)
I20260619 14:56:55.451834 16959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:55.505113 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.504529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39559 (local address 127.0.0.1:35998)
0619 14:56:55.504824 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.504927 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260619 14:56:55.506189 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.505858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39559 (local address 127.0.0.1:36010)
0619 14:56:55.505993 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.506079 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 14:56:55.562793 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 16837
I20260619 14:56:55.566793 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.566250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39559 (local address 127.0.0.1:36026)
0619 14:56:55.566474 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.566576 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 14:56:55.567806 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.567452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39559 (local address 127.0.0.1:36034)
0619 14:56:55.567604 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.567680 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 14:56:55.581125 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 16967
I20260619 14:56:55.583829 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.581836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46906 (local address 127.11.175.195:38697)
0619 14:56:55.582043 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:55.582048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:55.582065 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:56:55.582191 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:55.582195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:55.582268 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:55.582349 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:55.583283 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:55.583288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:55.583291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:55.583307 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:55.583340 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:55.583352 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:55.583461 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:55.583563 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 14:56:55.583611 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 14:56:55.584470 17089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:55.648376 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.647846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39559 (local address 127.0.0.1:36050)
0619 14:56:55.648104 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.648214 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260619 14:56:55.649459 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:55.649050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39559 (local address 127.0.0.1:36058)
0619 14:56:55.649272 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:55.649340 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260619 14:56:55.682746 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 16967
I20260619 14:56:55.699334 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 16616
I20260619 14:56:55.700664 16677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:55.808642 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 16616
2026-06-19T14:56:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2708 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 14:56:55.850909 11967 mini_oidc.cc:152] Starting JWKS server
I20260619 14:56:55.853505 11967 webserver.cc:460] Webserver started at https://127.0.0.1:42249/ using document root <none> and password file <none>
I20260619 14:56:55.853858 11967 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 14:56:55.854595 11967 webserver.cc:460] Webserver started at http://127.0.0.1:34453/ using document root <none> and password file <none>
2026-06-19T14:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:55Z Disabled control of system clock
I20260619 14:56:55.884524 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:34627
--webserver_interface=127.11.175.254
--webserver_port=0
--builtin_ntp_servers=127.11.175.212:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:34627
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42249/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/testchainca.cert with env {}
W20260619 14:56:56.216980 17150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:56.217346 17150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:56.217528 17150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:56.224982 17150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:56.225132 17150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:56.225286 17150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:56.225374 17150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:56.225415 17150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 14:56:56.225478 17150 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 14:56:56.225517 17150 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42249/jwks.json
I20260619 14:56:56.235556 17150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:34627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:34627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42249/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:56.236831 17150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:56.238112 17150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:56.250213 17155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:56.250298 17156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:56.250242 17158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:56.251366 17150 server_base.cc:1034] running on GCE node
I20260619 14:56:56.252229 17150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:56.253716 17150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:56.254988 17150 hybrid_clock.cc:648] HybridClock initialized: now 1781881016254965 us; error 64 us; skew 500 ppm
I20260619 14:56:56.255358 17150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:56:56.257802 17150 webserver.cc:460] Webserver started at http://127.11.175.254:33167/ using document root <none> and password file <none>
I20260619 14:56:56.258518 17150 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:56.258679 17150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:56.259024 17150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:56.261966 17150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "6337ff219ea54950964c3b3f52d010f6"
format_stamp: "Formatted at 2026-06-19 14:56:56 on dist-test-slave-76rk"
server_key: "55bc9d79b92021d3a05131a6e8ea98f9"
server_key_iv: "e4e81cd3724abe2a46d1caf33764504a"
server_key_version: "encryptionkey@0"
I20260619 14:56:56.262841 17150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "6337ff219ea54950964c3b3f52d010f6"
format_stamp: "Formatted at 2026-06-19 14:56:56 on dist-test-slave-76rk"
server_key: "55bc9d79b92021d3a05131a6e8ea98f9"
server_key_iv: "e4e81cd3724abe2a46d1caf33764504a"
server_key_version: "encryptionkey@0"
I20260619 14:56:56.268386 17150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 14:56:56.272323 17164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:56.273990 17150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 14:56:56.274202 17150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "6337ff219ea54950964c3b3f52d010f6"
format_stamp: "Formatted at 2026-06-19 14:56:56 on dist-test-slave-76rk"
server_key: "55bc9d79b92021d3a05131a6e8ea98f9"
server_key_iv: "e4e81cd3724abe2a46d1caf33764504a"
server_key_version: "encryptionkey@0"
I20260619 14:56:56.274451 17150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:56.295770 17150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:56.296715 17150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:56.297058 17150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:56.316120 17150 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:34627
I20260619 14:56:56.316143 17215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:34627 every 8 connection(s)
I20260619 14:56:56.317556 17150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:56.321480 17216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:56.324666 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 17150
I20260619 14:56:56.324908 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:56.325628 11967 external_mini_cluster.cc:1428] Setting key 7f96b753930a0bf98a7b1b8cc2c0b2d3
I20260619 14:56:56.335686 17216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6: Bootstrap starting.
I20260619 14:56:56.339613 17216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:56.340968 17216 log.cc:824] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:56.341871 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:56.327692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46558 (local address 127.11.175.254:34627)
0619 14:56:56.328770 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:56.328787 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:56.328839 (+ 52us) server_negotiation.cc:407] Connection header received
0619 14:56:56.330896 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:56.330934 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:56.331628 (+ 694us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:56.332138 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:56.332737 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:56.334736 (+ 1999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:56.336331 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:56.337388 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:56.337614 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:56.338882 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:56.338892 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:56.338911 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:56.339658 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:56.339789 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:56.339818 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:56.339933 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:56.340166 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 14:56:56.340512 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":328,"threads_started":1}
I20260619 14:56:56.344375 17216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6: No bootstrap required, opened a new log
I20260619 14:56:56.346693 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 14:56:56.352753 17216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337ff219ea54950964c3b3f52d010f6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34627 } }
I20260619 14:56:56.353155 17216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:56.353262 17216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6337ff219ea54950964c3b3f52d010f6, State: Initialized, Role: FOLLOWER
I20260619 14:56:56.353818 17216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337ff219ea54950964c3b3f52d010f6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34627 } }
I20260619 14:56:56.354070 17216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:56.354213 17216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:56.354377 17216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:56.357146 17216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337ff219ea54950964c3b3f52d010f6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34627 } }
I20260619 14:56:56.357710 17216 leader_election.cc:302] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6337ff219ea54950964c3b3f52d010f6; no voters:
I20260619 14:56:56.358331 17216 leader_election.cc:288] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:56.358592 17221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:56.359499 17221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [term 1 LEADER]: Becoming Leader. State: Replica: 6337ff219ea54950964c3b3f52d010f6, State: Running, Role: LEADER
I20260619 14:56:56.360034 17221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337ff219ea54950964c3b3f52d010f6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34627 } }
I20260619 14:56:56.360731 17216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:56.363189 17223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6337ff219ea54950964c3b3f52d010f6. Latest consensus state: current_term: 1 leader_uuid: "6337ff219ea54950964c3b3f52d010f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337ff219ea54950964c3b3f52d010f6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34627 } } }
I20260619 14:56:56.363673 17223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:56.365509 17230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:56.365306 17222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6337ff219ea54950964c3b3f52d010f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337ff219ea54950964c3b3f52d010f6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34627 } } }
I20260619 14:56:56.365697 17222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:56.370373 17230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:56.385252 17230 catalog_manager.cc:1347] Generated new cluster ID: 45f2ef8620d341498807363699666b33
I20260619 14:56:56.385613 17230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:56.395848 17230 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:56.397704 17230 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:56.404505 17230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6337ff219ea54950964c3b3f52d010f6: Generated new TSK 0
I20260619 14:56:56.405602 17230 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 14:56:56.440349 17246 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 14:56:56.440578 17246 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 14:56:56.442060 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:56.420793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.11.175.254:34627)
0619 14:56:56.420951 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:56.420956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:56.421417 (+ 461us) server_negotiation.cc:407] Connection header received
0619 14:56:56.421691 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:56.421697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:56.421797 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:56.421974 (+ 177us) server_negotiation.cc:226] Negotiated authn=JWT
0619 14:56:56.422489 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:56.423654 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:56.425155 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:56.425850 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:56.425997 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:56.426510 (+ 513us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 14:56:56.441312 (+ 14802us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 14:56:56.441762 (+ 450us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46564: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 14:56:56.442461 17219 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46564: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 14:56:56.444458 17245 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:56:56.420658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:34627 (local address 127.0.0.1:46564)
0619 14:56:56.421305 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:56.421326 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:56.421467 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:56.421996 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:56.422000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:56.422009 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0619 14:56:56.422322 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:56.422332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:56.423824 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:56.423828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:56.425011 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:56.425023 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:56.426050 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:56.426055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:56.426180 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:56.426237 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 14:56:56.441790 (+ 15553us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 14:56:56.442433 (+ 643us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.175.254:34627: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":326,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.175.254:34627: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 14:56:56.446666 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 17150
I20260619 14:56:56.447839 17211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:56.539822 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 17150
2026-06-19T14:56:56Z chronyd exiting
I20260619 14:56:56.590401 11967 test_util.cc:175] -----------------------------------------------
I20260619 14:56:56.590538 11967 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781880972997616-11967-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (744 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 14:56:56.594285 11967 mini_oidc.cc:152] Starting JWKS server
I20260619 14:56:56.596166 11967 webserver.cc:460] Webserver started at https://127.0.0.1:36011/ using document root <none> and password file <none>
I20260619 14:56:56.596426 11967 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 14:56:56.597085 11967 webserver.cc:460] Webserver started at http://127.0.0.1:42565/ using document root <none> and password file <none>
2026-06-19T14:56:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:56Z Disabled control of system clock
I20260619 14:56:56.624919 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:36843
--webserver_interface=127.11.175.254
--webserver_port=0
--builtin_ntp_servers=127.11.175.212:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:36843
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36011/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/testchainca.cert with env {}
W20260619 14:56:56.957716 17253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:56.958020 17253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:56.958181 17253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:56.965643 17253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:56.965752 17253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:56.965874 17253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:56.965972 17253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:56.966038 17253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 14:56:56.966130 17253 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 14:56:56.966184 17253 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36011/jwks.json
I20260619 14:56:56.977224 17253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:36843
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:36843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36011/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:56.978577 17253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:56.979985 17253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:56.991848 17258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:56.992054 17261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:56.992081 17259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:56.993613 17253 server_base.cc:1034] running on GCE node
I20260619 14:56:56.994556 17253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:56.996081 17253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:56.997354 17253 hybrid_clock.cc:648] HybridClock initialized: now 1781881016997288 us; error 56 us; skew 500 ppm
I20260619 14:56:56.997776 17253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:56:57.000214 17253 webserver.cc:460] Webserver started at http://127.11.175.254:37703/ using document root <none> and password file <none>
I20260619 14:56:57.000947 17253 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:57.001087 17253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:57.001422 17253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:57.004420 17253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "239595eb75df4f49b95300abc5fee47b"
format_stamp: "Formatted at 2026-06-19 14:56:56 on dist-test-slave-76rk"
server_key: "5babc827afa3083d51539852c4a2df93"
server_key_iv: "4e911a19b533cf182e1cca0bce28bbe7"
server_key_version: "encryptionkey@0"
I20260619 14:56:57.005214 17253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "239595eb75df4f49b95300abc5fee47b"
format_stamp: "Formatted at 2026-06-19 14:56:56 on dist-test-slave-76rk"
server_key: "5babc827afa3083d51539852c4a2df93"
server_key_iv: "4e911a19b533cf182e1cca0bce28bbe7"
server_key_version: "encryptionkey@0"
I20260619 14:56:57.010375 17253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 14:56:57.014292 17267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:57.016049 17253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:56:57.016270 17253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "239595eb75df4f49b95300abc5fee47b"
format_stamp: "Formatted at 2026-06-19 14:56:56 on dist-test-slave-76rk"
server_key: "5babc827afa3083d51539852c4a2df93"
server_key_iv: "4e911a19b533cf182e1cca0bce28bbe7"
server_key_version: "encryptionkey@0"
I20260619 14:56:57.016515 17253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:57.032546 17253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:57.033411 17253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:57.033737 17253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:57.052502 17253 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:36843
I20260619 14:56:57.052521 17318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:36843 every 8 connection(s)
I20260619 14:56:57.053929 17253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:57.057682 17319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:57.064291 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 17253
I20260619 14:56:57.064512 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:57.065352 11967 external_mini_cluster.cc:1428] Setting key 7181e20d858922177b79b278ee88f5b9
I20260619 14:56:57.069474 17319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b: Bootstrap starting.
I20260619 14:56:57.072955 17319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:57.074051 17319 log.cc:824] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:57.077492 17319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b: No bootstrap required, opened a new log
I20260619 14:56:57.079874 17322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:57.067411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.11.175.254:36843)
0619 14:56:57.068412 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:57.068428 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:57.068472 (+ 44us) server_negotiation.cc:407] Connection header received
0619 14:56:57.070274 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:57.070299 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:57.070860 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:57.071323 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:57.071922 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:57.073271 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:57.074742 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:57.075581 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:57.075776 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:57.076975 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:57.076987 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:57.077005 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:57.077725 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:57.077826 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:57.077851 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:57.077980 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:57.078217 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 14:56:57.078578 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":304,"threads_started":1}
I20260619 14:56:57.084175 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 14:56:57.086841 17319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239595eb75df4f49b95300abc5fee47b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36843 } }
I20260619 14:56:57.087270 17319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:57.087378 17319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239595eb75df4f49b95300abc5fee47b, State: Initialized, Role: FOLLOWER
I20260619 14:56:57.088013 17319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239595eb75df4f49b95300abc5fee47b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36843 } }
I20260619 14:56:57.088225 17319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:57.088325 17319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:57.088449 17319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:57.090683 17319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239595eb75df4f49b95300abc5fee47b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36843 } }
I20260619 14:56:57.091194 17319 leader_election.cc:302] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 239595eb75df4f49b95300abc5fee47b; no voters:
I20260619 14:56:57.091960 17319 leader_election.cc:288] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:57.092131 17324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:57.092974 17324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [term 1 LEADER]: Becoming Leader. State: Replica: 239595eb75df4f49b95300abc5fee47b, State: Running, Role: LEADER
I20260619 14:56:57.093485 17324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239595eb75df4f49b95300abc5fee47b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36843 } }
I20260619 14:56:57.094211 17319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:57.096966 17326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 239595eb75df4f49b95300abc5fee47b. Latest consensus state: current_term: 1 leader_uuid: "239595eb75df4f49b95300abc5fee47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239595eb75df4f49b95300abc5fee47b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36843 } } }
I20260619 14:56:57.097342 17326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:57.098526 17325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "239595eb75df4f49b95300abc5fee47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "239595eb75df4f49b95300abc5fee47b" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36843 } } }
I20260619 14:56:57.101466 17325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:57.102717 17329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:57.107080 17329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:57.117254 17329 catalog_manager.cc:1347] Generated new cluster ID: 448dc84d80e24b048aa8a4a0e3cdeb2c
I20260619 14:56:57.117408 17329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:57.159500 17329 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:57.160797 17329 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:57.170840 17329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 239595eb75df4f49b95300abc5fee47b: Generated new TSK 0
I20260619 14:56:57.171845 17329 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 14:56:57.210201 17349 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 14:56:57.210366 17349 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 14:56:57.211645 17322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:57.190168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.11.175.254:36843)
0619 14:56:57.190334 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:57.190339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:57.190823 (+ 484us) server_negotiation.cc:407] Connection header received
0619 14:56:57.191116 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:57.191124 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:57.191223 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:57.191403 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0619 14:56:57.191826 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:57.192979 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:57.194538 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:57.195282 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:57.195414 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:57.195979 (+ 565us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 14:56:57.210953 (+ 14974us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 14:56:57.211342 (+ 389us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34260: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 14:56:57.211773 17322 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34260: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 14:56:57.211838 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:56:57.190018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:36843 (local address 127.0.0.1:34260)
0619 14:56:57.190721 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:57.190738 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:57.190866 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:57.191426 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:57.191430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:57.191440 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0619 14:56:57.191680 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:57.191687 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:57.193129 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:57.193134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:57.194348 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:57.194361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:57.195449 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:57.195454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:57.195651 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:57.195689 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 14:56:57.211251 (+ 15562us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 14:56:57.211472 (+ 221us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.175.254:36843: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":381,"threads_started":1}
I20260619 14:56:57.213255 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 17253
I20260619 14:56:57.214430 17314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:57.311496 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 17253
2026-06-19T14:56:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (773 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 14:56:57.367967 11967 mini_oidc.cc:152] Starting JWKS server
I20260619 14:56:57.370085 11967 webserver.cc:460] Webserver started at https://127.0.0.1:41543/ using document root <none> and password file <none>
I20260619 14:56:57.370334 11967 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 14:56:57.371104 11967 webserver.cc:460] Webserver started at http://127.0.0.1:44017/ using document root <none> and password file <none>
2026-06-19T14:56:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:57Z Disabled control of system clock
I20260619 14:56:57.402736 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38679
--webserver_interface=127.11.175.254
--webserver_port=0
--builtin_ntp_servers=127.11.175.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38679
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41543/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 14:56:57.734773 17356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:57.735148 17356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:57.735306 17356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:57.743062 17356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:57.743216 17356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:57.743306 17356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:57.743422 17356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:57.743505 17356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 14:56:57.743597 17356 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 14:56:57.743674 17356 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41543/jwks.json
I20260619 14:56:57.753203 17356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38679
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41543/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:57.754565 17356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:57.756446 17356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:57.768745 17364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:57.769018 17361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:57.768994 17362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:57.769757 17356 server_base.cc:1034] running on GCE node
I20260619 14:56:57.770772 17356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:57.772567 17356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:57.773825 17356 hybrid_clock.cc:648] HybridClock initialized: now 1781881017773758 us; error 61 us; skew 500 ppm
I20260619 14:56:57.774287 17356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:56:57.776846 17356 webserver.cc:460] Webserver started at http://127.11.175.254:35471/ using document root <none> and password file <none>
I20260619 14:56:57.777580 17356 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:57.777724 17356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:57.778074 17356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:57.781054 17356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "43b20dcac7d9490ea9584e0094a99635"
format_stamp: "Formatted at 2026-06-19 14:56:57 on dist-test-slave-76rk"
server_key: "f0ad8c6ea70b321ac08bc07e3dcd7137"
server_key_iv: "0d15de22faa81548a7fb6b29f70f2edd"
server_key_version: "encryptionkey@0"
I20260619 14:56:57.781838 17356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "43b20dcac7d9490ea9584e0094a99635"
format_stamp: "Formatted at 2026-06-19 14:56:57 on dist-test-slave-76rk"
server_key: "f0ad8c6ea70b321ac08bc07e3dcd7137"
server_key_iv: "0d15de22faa81548a7fb6b29f70f2edd"
server_key_version: "encryptionkey@0"
I20260619 14:56:57.787169 17356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 14:56:57.790738 17370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:57.792253 17356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:57.792474 17356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "43b20dcac7d9490ea9584e0094a99635"
format_stamp: "Formatted at 2026-06-19 14:56:57 on dist-test-slave-76rk"
server_key: "f0ad8c6ea70b321ac08bc07e3dcd7137"
server_key_iv: "0d15de22faa81548a7fb6b29f70f2edd"
server_key_version: "encryptionkey@0"
I20260619 14:56:57.792711 17356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:57.814529 17356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:57.815507 17356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:57.815856 17356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:57.835644 17356 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:38679
I20260619 14:56:57.835651 17421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:38679 every 8 connection(s)
I20260619 14:56:57.837153 17356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:57.841193 17422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:57.844492 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 17356
I20260619 14:56:57.844736 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:57.845515 11967 external_mini_cluster.cc:1428] Setting key da87a6448d211830eaa1ea5417e75b1d
I20260619 14:56:57.864200 17422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635: Bootstrap starting.
I20260619 14:56:57.868340 17422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:57.869845 17422 log.cc:824] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:57.871651 17425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:57.847781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54986 (local address 127.11.175.254:38679)
0619 14:56:57.856200 (+ 8419us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:57.856217 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:57.856275 (+ 58us) server_negotiation.cc:407] Connection header received
0619 14:56:57.859061 (+ 2786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:57.859106 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:57.859970 (+ 864us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:57.860488 (+ 518us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:57.861191 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:57.863185 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:57.865588 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:57.866798 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:57.866993 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:57.868478 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:57.868491 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:57.868512 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:57.869287 (+ 775us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:57.869428 (+ 141us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:57.869496 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:57.869613 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:57.869931 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 14:56:57.870326 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7780,"thread_start_us":7522,"threads_started":1}
I20260619 14:56:57.873872 17422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635: No bootstrap required, opened a new log
I20260619 14:56:57.876643 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 14:56:57.883390 17422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b20dcac7d9490ea9584e0094a99635" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38679 } }
I20260619 14:56:57.883821 17422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:57.883950 17422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b20dcac7d9490ea9584e0094a99635, State: Initialized, Role: FOLLOWER
I20260619 14:56:57.884670 17422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b20dcac7d9490ea9584e0094a99635" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38679 } }
I20260619 14:56:57.884992 17422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:57.885160 17422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:57.885360 17422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:57.887827 17422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b20dcac7d9490ea9584e0094a99635" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38679 } }
I20260619 14:56:57.888473 17422 leader_election.cc:302] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43b20dcac7d9490ea9584e0094a99635; no voters:
I20260619 14:56:57.889191 17422 leader_election.cc:288] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:57.889330 17427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:57.892381 17422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:57.897181 17427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [term 1 LEADER]: Becoming Leader. State: Replica: 43b20dcac7d9490ea9584e0094a99635, State: Running, Role: LEADER
I20260619 14:56:57.897987 17427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b20dcac7d9490ea9584e0094a99635" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38679 } }
I20260619 14:56:57.901075 17428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43b20dcac7d9490ea9584e0094a99635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b20dcac7d9490ea9584e0094a99635" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38679 } } }
I20260619 14:56:57.901569 17428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:57.902242 17429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b20dcac7d9490ea9584e0094a99635. Latest consensus state: current_term: 1 leader_uuid: "43b20dcac7d9490ea9584e0094a99635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b20dcac7d9490ea9584e0094a99635" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38679 } } }
I20260619 14:56:57.902618 17429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635 [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:57.911419 17437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:57.918200 17437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:57.929436 17437 catalog_manager.cc:1347] Generated new cluster ID: 617923bacd1f4a4295931daab1b8b96f
I20260619 14:56:57.929575 17437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:57.956825 17437 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:57.958777 17437 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:57.969295 17437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43b20dcac7d9490ea9584e0094a99635: Generated new TSK 0
I20260619 14:56:57.970418 17437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 14:56:58.008242 17425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:57.997565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55000 (local address 127.11.175.254:38679)
0619 14:56:57.997836 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:57.997848 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:57.998515 (+ 667us) server_negotiation.cc:407] Connection header received
0619 14:56:57.998805 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:57.998812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:57.998915 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:57.999077 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0619 14:56:57.999704 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:58.000912 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:58.003437 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:58.004197 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:58.004329 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:58.005034 (+ 705us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 14:56:58.007518 (+ 2484us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41543/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 14:56:58.007866 (+ 348us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55000: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41543/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":102}
W20260619 14:56:58.008378 17425 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55000: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41543/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 14:56:58.009634 17451 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:56:57.997352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38679 (local address 127.0.0.1:55000)
0619 14:56:57.998372 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:57.998394 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:57.998585 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:57.999112 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:57.999121 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:57.999139 (+ 18us) client_negotiation.cc:189] Negotiated authn=JWT
0619 14:56:57.999528 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:57.999545 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:58.001069 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:58.001078 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:58.003264 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:58.003283 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:58.004433 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:58.004442 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:58.004679 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:58.004742 (+ 63us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 14:56:58.008856 (+ 4114us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41543/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 14:56:58.009157 (+ 301us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.175.254:38679: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41543/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":662,"threads_started":1}
I20260619 14:56:58.010830 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 17356
I20260619 14:56:58.012485 17417 generic_service.cc:196] Checking for leaks (request via RPC)
E20260619 14:56:58.032161 17452 webserver.cc:533] Webserver: sslize failed:
E20260619 14:56:58.032311 17452 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 14:56:58.112483 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 17356
2026-06-19T14:56:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (801 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T14:56:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:56:58Z Disabled control of system clock
I20260619 14:56:58.197544 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41273
--webserver_interface=127.11.175.254
--webserver_port=0
--builtin_ntp_servers=127.11.175.212:43833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:41273
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/testchainca.cert with env {}
W20260619 14:56:58.532141 17457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:56:58.532454 17457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:56:58.532542 17457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:56:58.540102 17457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:56:58.540223 17457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:56:58.540266 17457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:56:58.540316 17457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:56:58.540354 17457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 14:56:58.540416 17457 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 14:56:58.540457 17457 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 14:56:58.550870 17457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:41273
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:56:58.552207 17457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:56:58.553496 17457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:56:58.566164 17462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:58.566164 17465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:56:58.566192 17463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:56:58.567131 17457 server_base.cc:1034] running on GCE node
I20260619 14:56:58.568102 17457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:56:58.569613 17457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:56:58.570815 17457 hybrid_clock.cc:648] HybridClock initialized: now 1781881018570713 us; error 104 us; skew 500 ppm
I20260619 14:56:58.571202 17457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:56:58.573757 17457 webserver.cc:460] Webserver started at http://127.11.175.254:41245/ using document root <none> and password file <none>
I20260619 14:56:58.574529 17457 fs_manager.cc:359] Metadata directory not provided
I20260619 14:56:58.574684 17457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:56:58.575031 17457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:56:58.578186 17457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "66288ecb33b54793952bf2f9432f90eb"
format_stamp: "Formatted at 2026-06-19 14:56:58 on dist-test-slave-76rk"
server_key: "d766b725d29a2b68d3359abc33d60cae"
server_key_iv: "05b3ad214cc0ce83f5ac0ccdbd53038c"
server_key_version: "encryptionkey@0"
I20260619 14:56:58.579077 17457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "66288ecb33b54793952bf2f9432f90eb"
format_stamp: "Formatted at 2026-06-19 14:56:58 on dist-test-slave-76rk"
server_key: "d766b725d29a2b68d3359abc33d60cae"
server_key_iv: "05b3ad214cc0ce83f5ac0ccdbd53038c"
server_key_version: "encryptionkey@0"
I20260619 14:56:58.584447 17457 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 14:56:58.588145 17471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:56:58.589774 17457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:56:58.590032 17457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "66288ecb33b54793952bf2f9432f90eb"
format_stamp: "Formatted at 2026-06-19 14:56:58 on dist-test-slave-76rk"
server_key: "d766b725d29a2b68d3359abc33d60cae"
server_key_iv: "05b3ad214cc0ce83f5ac0ccdbd53038c"
server_key_version: "encryptionkey@0"
I20260619 14:56:58.590278 17457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:56:58.623306 17457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:56:58.624209 17457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:56:58.624526 17457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:56:58.644191 17457 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:41273
I20260619 14:56:58.644232 17522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:41273 every 8 connection(s)
I20260619 14:56:58.645599 17457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:56:58.649495 17523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:56:58.649806 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 17457
I20260619 14:56:58.650007 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:56:58.650712 11967 external_mini_cluster.cc:1428] Setting key fd4c9d0ff8b00142f91fb09619fc2684
I20260619 14:56:58.662099 17523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb: Bootstrap starting.
I20260619 14:56:58.664819 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:58.652742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.11.175.254:41273)
0619 14:56:58.653796 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:58.653814 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:58.653860 (+ 46us) server_negotiation.cc:407] Connection header received
0619 14:56:58.655907 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:58.655934 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:58.656456 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:58.656891 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:56:58.657457 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:58.658784 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:58.660114 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:58.660812 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:58.661013 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:58.662286 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:56:58.662299 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:56:58.662319 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:56:58.662969 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:56:58.663052 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:56:58.663074 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:56:58.663213 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 14:56:58.663453 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 14:56:58.663784 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":356,"threads_started":1}
I20260619 14:56:58.665751 17523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb: Neither blocks nor log segments found. Creating new log.
I20260619 14:56:58.666991 17523 log.cc:824] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb: Log is configured to *not* fsync() on all Append() calls
I20260619 14:56:58.669118 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 14:56:58.670401 17523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb: No bootstrap required, opened a new log
I20260619 14:56:58.678220 17523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66288ecb33b54793952bf2f9432f90eb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41273 } }
I20260619 14:56:58.678588 17523 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:56:58.678716 17523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66288ecb33b54793952bf2f9432f90eb, State: Initialized, Role: FOLLOWER
I20260619 14:56:58.679224 17523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66288ecb33b54793952bf2f9432f90eb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41273 } }
I20260619 14:56:58.679426 17523 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:56:58.679527 17523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:56:58.679646 17523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:56:58.681741 17523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66288ecb33b54793952bf2f9432f90eb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41273 } }
I20260619 14:56:58.682339 17523 leader_election.cc:302] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66288ecb33b54793952bf2f9432f90eb; no voters:
I20260619 14:56:58.683037 17523 leader_election.cc:288] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:56:58.683228 17528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:56:58.684032 17528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [term 1 LEADER]: Becoming Leader. State: Replica: 66288ecb33b54793952bf2f9432f90eb, State: Running, Role: LEADER
I20260619 14:56:58.684700 17528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66288ecb33b54793952bf2f9432f90eb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41273 } }
I20260619 14:56:58.685233 17523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:56:58.687865 17529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66288ecb33b54793952bf2f9432f90eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66288ecb33b54793952bf2f9432f90eb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41273 } } }
I20260619 14:56:58.687878 17530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66288ecb33b54793952bf2f9432f90eb. Latest consensus state: current_term: 1 leader_uuid: "66288ecb33b54793952bf2f9432f90eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66288ecb33b54793952bf2f9432f90eb" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41273 } } }
I20260619 14:56:58.688393 17529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [sys.catalog]: This master's current role is: LEADER
I20260619 14:56:58.688482 17530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb [sys.catalog]: This master's current role is: LEADER
W20260619 14:56:58.696427 17543 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 14:56:58.696557 17543 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 14:56:58.696713 17537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:56:58.699154 17537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:56:58.708513 17537 catalog_manager.cc:1347] Generated new cluster ID: 24280eca5c20456f8b1c5e3053185cf4
I20260619 14:56:58.708657 17537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:56:58.723892 17537 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:56:58.725940 17537 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:56:58.746762 17537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66288ecb33b54793952bf2f9432f90eb: Generated new TSK 0
I20260619 14:56:58.747978 17537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 14:56:58.807796 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:56:58.764743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.11.175.254:41273)
0619 14:56:58.764967 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 14:56:58.764978 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:56:58.765385 (+ 407us) server_negotiation.cc:407] Connection header received
0619 14:56:58.765619 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:56:58.765625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:56:58.765720 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:56:58.765913 (+ 193us) server_negotiation.cc:226] Negotiated authn=JWT
0619 14:56:58.766343 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:58.767521 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:58.769070 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:56:58.769782 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:56:58.769929 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:58.770437 (+ 508us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 14:56:58.807008 (+ 36571us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 14:56:58.807466 (+ 458us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42334: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 14:56:58.807962 17526 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42334: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 14:56:58.808091 17552 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:56:58.764606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41273 (local address 127.0.0.1:42334)
0619 14:56:58.765276 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0619 14:56:58.765292 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:56:58.765437 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:56:58.765930 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:56:58.765935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:56:58.765948 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0619 14:56:58.766191 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:58.766199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:58.767684 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:58.767688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:58.768899 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:56:58.768913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:56:58.769983 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:56:58.769988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:56:58.770110 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:56:58.770144 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 14:56:58.807317 (+ 37173us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 14:56:58.807634 (+ 317us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.175.254:41273: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":387,"threads_started":1}
I20260619 14:56:58.809348 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 17457
I20260619 14:56:58.810747 17518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:56:58.909781 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 17457
2026-06-19T14:56:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (776 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (424 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (423 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:56:59 dist-test-slave-76rk krb5kdc[17567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:56:59 dist-test-slave-76rk krb5kdc[17567](info): set up 2 sockets
Jun 19 14:56:59 dist-test-slave-76rk krb5kdc[17567](info): commencing operation
krb5kdc: starting...
W20260619 14:57:00.895778 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:57:01 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:57:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:57:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:57:01.131712 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:35707
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:35707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:01.479019 17580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:01.479334 17580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:01.479506 17580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:01.487358 17580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:01.487498 17580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:01.487596 17580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:01.487715 17580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:01.487799 17580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:01.498463 17580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:35707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:35707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:01.499997 17580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:01.501457 17580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:01.514475 17586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:01.514423 17588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:01.514762 17585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:01.515373 17580 server_base.cc:1034] running on GCE node
I20260619 14:57:01.516187 17580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:01.517724 17580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:01.518940 17580 hybrid_clock.cc:648] HybridClock initialized: now 1781881021518888 us; error 36 us; skew 500 ppm
I20260619 14:57:01.519269 17580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:01 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881021, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:01.522581 17580 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:57:01.524330 17580 webserver.cc:460] Webserver started at http://127.11.175.254:40233/ using document root <none> and password file <none>
I20260619 14:57:01.525025 17580 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:01.525132 17580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:01.525398 17580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:01.528520 17580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "57f5fa3704254910a01220542816fed9"
format_stamp: "Formatted at 2026-06-19 14:57:01 on dist-test-slave-76rk"
server_key: "dec84fe323efdebd5546506c7ae784a5"
server_key_iv: "90e87324cc27e032beb28644aca21442"
server_key_version: "encryptionkey@0"
I20260619 14:57:01.529254 17580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "57f5fa3704254910a01220542816fed9"
format_stamp: "Formatted at 2026-06-19 14:57:01 on dist-test-slave-76rk"
server_key: "dec84fe323efdebd5546506c7ae784a5"
server_key_iv: "90e87324cc27e032beb28644aca21442"
server_key_version: "encryptionkey@0"
I20260619 14:57:01.535084 17580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 14:57:01.538888 17595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:01.540467 17580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:57:01.540689 17580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "57f5fa3704254910a01220542816fed9"
format_stamp: "Formatted at 2026-06-19 14:57:01 on dist-test-slave-76rk"
server_key: "dec84fe323efdebd5546506c7ae784a5"
server_key_iv: "90e87324cc27e032beb28644aca21442"
server_key_version: "encryptionkey@0"
I20260619 14:57:01.540907 17580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:01.568720 17580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:01.573628 17580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:01.574030 17580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:01.593871 17580 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:35707
I20260619 14:57:01.593936 17646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:35707 every 8 connection(s)
I20260619 14:57:01.595396 17580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:57:01.599382 17647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:01.604904 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 17580
I20260619 14:57:01.605157 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:57:01.605795 11967 external_mini_cluster.cc:1428] Setting key f4e265c909c5f4977f6c7a4650cdae8f
I20260619 14:57:01.610927 17647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9: Bootstrap starting.
I20260619 14:57:01.615195 17647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:01.616619 17647 log.cc:824] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:01.620386 17647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9: No bootstrap required, opened a new log
Jun 19 14:57:01 dist-test-slave-76rk krb5kdc[17567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:01.629009 17647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f5fa3704254910a01220542816fed9" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35707 } }
I20260619 14:57:01.629529 17647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:01.629675 17647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f5fa3704254910a01220542816fed9, State: Initialized, Role: FOLLOWER
I20260619 14:57:01.630414 17647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f5fa3704254910a01220542816fed9" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35707 } }
I20260619 14:57:01.630796 17647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:57:01.630965 17647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:57:01.631181 17647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:01.632081 17650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:01.608015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40992 (local address 127.11.175.254:35707)
0619 14:57:01.608907 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:01.608926 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:01.608975 (+ 49us) server_negotiation.cc:407] Connection header received
0619 14:57:01.610803 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:01.610828 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:01.611304 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:01.611667 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:01.613953 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:01.615496 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:01.617323 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:01.618407 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:01.618550 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:01.623618 (+ 5068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:01.623627 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:01.623644 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:01.623689 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:01.627195 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:01.628079 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:01.628087 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:01.628096 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:01.628311 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:01.628734 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:01.628739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:01.628743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:01.629360 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:01.629709 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:01.630195 (+ 486us) server_negotiation.cc:299] Negotiation successful
0619 14:57:01.630738 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":431,"threads_started":1}
I20260619 14:57:01.633721 17647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f5fa3704254910a01220542816fed9" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35707 } }
I20260619 14:57:01.634308 17647 leader_election.cc:302] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57f5fa3704254910a01220542816fed9; no voters:
I20260619 14:57:01.635044 17647 leader_election.cc:288] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:57:01.635144 17652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:01.635991 17652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [term 1 LEADER]: Becoming Leader. State: Replica: 57f5fa3704254910a01220542816fed9, State: Running, Role: LEADER
I20260619 14:57:01.636595 17652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f5fa3704254910a01220542816fed9" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35707 } }
I20260619 14:57:01.637575 17647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:57:01.640643 17654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57f5fa3704254910a01220542816fed9. Latest consensus state: current_term: 1 leader_uuid: "57f5fa3704254910a01220542816fed9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f5fa3704254910a01220542816fed9" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35707 } } }
I20260619 14:57:01.641206 17654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:01.640620 17653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57f5fa3704254910a01220542816fed9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f5fa3704254910a01220542816fed9" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35707 } } }
I20260619 14:57:01.641634 17653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:01.651579 17660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:57:01.657927 17660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:57:01.667850 17660 catalog_manager.cc:1347] Generated new cluster ID: b5a808a8db14480c8320b9216da30d59
I20260619 14:57:01.668030 17660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:57:01.698220 17660 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:57:01.700368 17660 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:57:01.715689 17660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57f5fa3704254910a01220542816fed9: Generated new TSK 0
I20260619 14:57:01.716683 17660 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:57:01.791397 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35707
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:02.151636 17673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:02.151983 17673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:02.152151 17673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:02.159850 17673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:02.159976 17673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:02.160094 17673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:57:02.170403 17673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:02.171898 17673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:02.173338 17673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:02.187858 17681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:02.188019 17678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:02.187955 17679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:02.189188 17673 server_base.cc:1034] running on GCE node
I20260619 14:57:02.189940 17673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:02.191112 17673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:02.192363 17673 hybrid_clock.cc:648] HybridClock initialized: now 1781881022192280 us; error 69 us; skew 500 ppm
I20260619 14:57:02.192766 17673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:02 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881022, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:02.196604 17673 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:57:02.198396 17673 webserver.cc:460] Webserver started at http://127.11.175.193:35935/ using document root <none> and password file <none>
I20260619 14:57:02.199199 17673 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:02.199311 17673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:02.199591 17673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:02.202818 17673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "fd5fdbf2a89742df9715a77ea348284d"
format_stamp: "Formatted at 2026-06-19 14:57:02 on dist-test-slave-76rk"
server_key: "4dd08197022d61ab4160bf42dfbb8eb3"
server_key_iv: "1ceeb835ed57bbbdd0a11459ac595c12"
server_key_version: "encryptionkey@0"
I20260619 14:57:02.203663 17673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "fd5fdbf2a89742df9715a77ea348284d"
format_stamp: "Formatted at 2026-06-19 14:57:02 on dist-test-slave-76rk"
server_key: "4dd08197022d61ab4160bf42dfbb8eb3"
server_key_iv: "1ceeb835ed57bbbdd0a11459ac595c12"
server_key_version: "encryptionkey@0"
I20260619 14:57:02.209362 17673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:57:02.213326 17688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:02.215008 17673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 14:57:02.215194 17673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "fd5fdbf2a89742df9715a77ea348284d"
format_stamp: "Formatted at 2026-06-19 14:57:02 on dist-test-slave-76rk"
server_key: "4dd08197022d61ab4160bf42dfbb8eb3"
server_key_iv: "1ceeb835ed57bbbdd0a11459ac595c12"
server_key_version: "encryptionkey@0"
I20260619 14:57:02.215435 17673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:02.231427 17673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:02.236433 17673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:02.236761 17673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:02.237875 17673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:02.239494 17673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:02.239599 17673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:02.239789 17673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:02.239864 17673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:02.275619 17673 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:46105
I20260619 14:57:02.275785 17800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:46105 every 8 connection(s)
I20260619 14:57:02.276998 17673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:57:02.286409 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 17673
I20260619 14:57:02.286669 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:57:02.287387 11967 external_mini_cluster.cc:1428] Setting key 67faabbd28074b816b4a9568f591a499
Jun 19 14:57:02 dist-test-slave-76rk krb5kdc[17567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881022, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:02.301519 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:02.279911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:59551 (local address 127.11.175.254:35707)
0619 14:57:02.280464 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:02.280469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:02.281241 (+ 772us) server_negotiation.cc:407] Connection header received
0619 14:57:02.283368 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:02.283374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:02.283488 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:02.283623 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:02.285445 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:02.286586 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:02.288775 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:02.289499 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:02.289664 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:02.294006 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:02.294016 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:02.294024 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:02.294074 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:02.297501 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:02.298486 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:02.298495 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:02.298502 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:02.298694 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:02.299260 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:02.299268 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:02.299274 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:02.299705 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:02.299856 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:02.300728 (+ 872us) server_negotiation.cc:299] Negotiation successful
0619 14:57:02.301020 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":288,"threads_started":1}
I20260619 14:57:02.302412 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:02.280290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35707 (local address 127.11.175.193:59551)
0619 14:57:02.281062 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:02.281103 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:02.283049 (+ 1946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:02.283901 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:02.283910 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:02.284493 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:02.285249 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:02.285274 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:02.286842 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:02.286849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:02.288589 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:02.288634 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:02.289805 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:02.289814 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:02.290102 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:02.290935 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:02.290963 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:02.293679 (+ 2716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:02.297701 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:02.297713 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:02.297734 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:02.298325 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:02.298857 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:02.298861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:02.298863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:02.299121 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:02.299879 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:02.299886 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:02.300269 (+ 383us) client_negotiation.cc:769] Sending connection context
0619 14:57:02.300583 (+ 314us) client_negotiation.cc:240] Negotiation successful
0619 14:57:02.301060 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":297,"threads_started":1}
I20260619 14:57:02.304405 17801 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35707
I20260619 14:57:02.304768 17801 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:02.305573 17801 heartbeater.cc:511] Master 127.11.175.254:35707 requested a full tablet report, sending...
I20260619 14:57:02.308357 17612 ts_manager.cc:194] Registered new tserver with Master: fd5fdbf2a89742df9715a77ea348284d (127.11.175.193:46105)
I20260619 14:57:02.310801 17612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:59551
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:57:02.342176 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35707
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:02.684553 17807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:02.684849 17807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:02.685015 17807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:02.692512 17807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:02.692641 17807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:02.692827 17807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:57:02.703053 17807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:02.704416 17807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:02.705773 17807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:02.720011 17813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:02.720312 17815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:02.720570 17812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:02.721105 17807 server_base.cc:1034] running on GCE node
I20260619 14:57:02.721836 17807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:02.722980 17807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:02.724195 17807 hybrid_clock.cc:648] HybridClock initialized: now 1781881022724149 us; error 40 us; skew 500 ppm
I20260619 14:57:02.724532 17807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:02 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881022, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:02.727905 17807 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:57:02.729537 17807 webserver.cc:460] Webserver started at http://127.11.175.194:43077/ using document root <none> and password file <none>
I20260619 14:57:02.730265 17807 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:02.730417 17807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:02.730892 17807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:02.733973 17807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "d64a1f5433a04b46be1c355fd3b71d86"
format_stamp: "Formatted at 2026-06-19 14:57:02 on dist-test-slave-76rk"
server_key: "31dfc937ccdda7320aa0e47b3c71c7ba"
server_key_iv: "3cb6bca6268524ce01daa261e42c00f8"
server_key_version: "encryptionkey@0"
I20260619 14:57:02.734871 17807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "d64a1f5433a04b46be1c355fd3b71d86"
format_stamp: "Formatted at 2026-06-19 14:57:02 on dist-test-slave-76rk"
server_key: "31dfc937ccdda7320aa0e47b3c71c7ba"
server_key_iv: "3cb6bca6268524ce01daa261e42c00f8"
server_key_version: "encryptionkey@0"
I20260619 14:57:02.740440 17807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 14:57:02.744632 17822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:02.746430 17807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 14:57:02.746589 17807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "d64a1f5433a04b46be1c355fd3b71d86"
format_stamp: "Formatted at 2026-06-19 14:57:02 on dist-test-slave-76rk"
server_key: "31dfc937ccdda7320aa0e47b3c71c7ba"
server_key_iv: "3cb6bca6268524ce01daa261e42c00f8"
server_key_version: "encryptionkey@0"
I20260619 14:57:02.746845 17807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:02.764499 17807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:02.769474 17807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:02.770169 17807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:02.771445 17807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:02.772892 17807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:02.772974 17807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:02.773070 17807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:02.773115 17807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:02.808183 17807 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:46225
I20260619 14:57:02.808228 17934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:46225 every 8 connection(s)
I20260619 14:57:02.809522 17807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:57:02.815766 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 17807
I20260619 14:57:02.816022 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:57:02.816483 11967 external_mini_cluster.cc:1428] Setting key 1bf5e31de6f78d18208ace51165bed90
Jun 19 14:57:02 dist-test-slave-76rk krb5kdc[17567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881022, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:02.831651 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:02.812136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:52281 (local address 127.11.175.254:35707)
0619 14:57:02.812603 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:02.812608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:02.813571 (+ 963us) server_negotiation.cc:407] Connection header received
0619 14:57:02.815558 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:02.815564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:02.815669 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:02.815792 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:02.817511 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:02.818301 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:02.820305 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:02.820799 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:02.820950 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:02.824795 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:02.824801 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:02.824804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:02.824843 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:02.827771 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:02.828670 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:02.828675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:02.828677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:02.828897 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:02.829555 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:02.829559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:02.829561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:02.829882 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:02.830019 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:02.831140 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0619 14:57:02.831359 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":243,"threads_started":1}
I20260619 14:57:02.833194 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:02.812649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35707 (local address 127.11.175.194:52281)
0619 14:57:02.813403 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:02.813441 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:02.815243 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:02.816051 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:02.816059 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:02.816692 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:02.817306 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:02.817325 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:02.818482 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:02.818487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:02.820127 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:02.820143 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:02.821072 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:02.821080 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:02.821328 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:02.822068 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:02.822098 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:02.824516 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:02.827969 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:02.827978 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:02.827999 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:02.828506 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:02.829067 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:02.829075 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:02.829082 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:02.829381 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:02.830037 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:02.830048 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:02.830614 (+ 566us) client_negotiation.cc:769] Sending connection context
0619 14:57:02.830966 (+ 352us) client_negotiation.cc:240] Negotiation successful
0619 14:57:02.831612 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":331,"threads_started":1}
I20260619 14:57:02.835080 17935 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35707
I20260619 14:57:02.835428 17935 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:02.836265 17935 heartbeater.cc:511] Master 127.11.175.254:35707 requested a full tablet report, sending...
I20260619 14:57:02.838091 17612 ts_manager.cc:194] Registered new tserver with Master: d64a1f5433a04b46be1c355fd3b71d86 (127.11.175.194:46225)
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
I20260619 14:57:02.839172 17612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:52281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:57:02.868604 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35707
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:03.210966 17941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:03.211324 17941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:03.211505 17941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:03.219101 17941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:03.219278 17941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:03.219455 17941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:57:03.229756 17941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:03.231333 17941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:03.232712 17941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:03.247408 17947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:03.247370 17946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:03.247534 17949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:03.248396 17941 server_base.cc:1034] running on GCE node
I20260619 14:57:03.249112 17941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:03.250200 17941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:03.251525 17941 hybrid_clock.cc:648] HybridClock initialized: now 1781881023251480 us; error 40 us; skew 500 ppm
I20260619 14:57:03.251952 17941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:03.255458 17941 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:57:03.257095 17941 webserver.cc:460] Webserver started at http://127.11.175.195:34503/ using document root <none> and password file <none>
I20260619 14:57:03.257902 17941 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:03.258060 17941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:03.258380 17941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:03.261539 17941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "518b60050f644a8d9128eb9842b9d41d"
format_stamp: "Formatted at 2026-06-19 14:57:03 on dist-test-slave-76rk"
server_key: "63b7be535c126c120b99200203255d85"
server_key_iv: "c6f7e1706ae4aab7a249bedcb5730cee"
server_key_version: "encryptionkey@0"
I20260619 14:57:03.262385 17941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "518b60050f644a8d9128eb9842b9d41d"
format_stamp: "Formatted at 2026-06-19 14:57:03 on dist-test-slave-76rk"
server_key: "63b7be535c126c120b99200203255d85"
server_key_iv: "c6f7e1706ae4aab7a249bedcb5730cee"
server_key_version: "encryptionkey@0"
I20260619 14:57:03.267987 17941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 14:57:03.272162 17956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:03.273624 17941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260619 14:57:03.273828 17941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "518b60050f644a8d9128eb9842b9d41d"
format_stamp: "Formatted at 2026-06-19 14:57:03 on dist-test-slave-76rk"
server_key: "63b7be535c126c120b99200203255d85"
server_key_iv: "c6f7e1706ae4aab7a249bedcb5730cee"
server_key_version: "encryptionkey@0"
I20260619 14:57:03.274106 17941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:03.314314 17801 heartbeater.cc:503] Master 127.11.175.254:35707 was elected leader, sending a full tablet report...
I20260619 14:57:03.329910 17941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:03.334609 17941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:03.334986 17941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:03.336045 17941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:03.337484 17941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:03.337555 17941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:03.337743 17941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:03.337793 17941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:03.375365 17941 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:33177
I20260619 14:57:03.375509 18068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:33177 every 8 connection(s)
I20260619 14:57:03.376762 17941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:57:03.383353 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 17941
I20260619 14:57:03.383592 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:57:03.384189 11967 external_mini_cluster.cc:1428] Setting key 499d94797638463821b30a28290f77af
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:03.400521 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:03.379622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:36865 (local address 127.11.175.254:35707)
0619 14:57:03.380132 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:03.380137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:03.381236 (+ 1099us) server_negotiation.cc:407] Connection header received
0619 14:57:03.383624 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:03.383630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:03.383768 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:03.383911 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:03.386228 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.387159 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.389029 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.389446 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.389587 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:03.393623 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:03.393630 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:03.393635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:03.393671 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:03.396473 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.397375 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.397381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.397383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.397536 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.398057 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.398063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.398065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.398518 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:03.398741 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:03.399776 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0619 14:57:03.400116 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":267,"threads_started":1}
I20260619 14:57:03.401736 18072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:03.380260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35707 (local address 127.11.175.195:36865)
0619 14:57:03.381029 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:03.381076 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:03.383002 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:03.384239 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:03.384256 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:03.385081 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:03.386008 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:03.386042 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.387350 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.387355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:03.388856 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:03.388869 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.389700 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.389707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:03.389975 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:03.390788 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:03.390814 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:03.393363 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:03.396643 (+ 3280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:03.396651 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:03.396666 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:03.397187 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:03.397657 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:03.397660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:03.397662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:03.397906 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:03.398764 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:03.398772 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:03.399201 (+ 429us) client_negotiation.cc:769] Sending connection context
0619 14:57:03.399595 (+ 394us) client_negotiation.cc:240] Negotiation successful
0619 14:57:03.400237 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":323,"threads_started":1}
I20260619 14:57:03.403486 18069 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35707
I20260619 14:57:03.403855 18069 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:03.404634 18069 heartbeater.cc:511] Master 127.11.175.254:35707 requested a full tablet report, sending...
I20260619 14:57:03.406307 17612 ts_manager.cc:194] Registered new tserver with Master: 518b60050f644a8d9128eb9842b9d41d (127.11.175.195:33177)
I20260619 14:57:03.407218 17612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:36865
I20260619 14:57:03.413723 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/test-admin.keytab.
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:03.442111 11967 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 14:57:03.447927 18077 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:03.445149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55254 (local address 127.11.175.193:46105)
0619 14:57:03.445865 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:03.445872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:03.445899 (+ 27us) server_negotiation.cc:407] Connection header received
0619 14:57:03.445962 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:03.445969 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:03.446199 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:03.446330 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:03.447459 (+ 1129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55254: BlockingRecv error: recv got EOF from 127.0.0.1:55254 (error 108)
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":388,"threads_started":1}
W20260619 14:57:03.447903 11967 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5cc' of type FILE
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:03.449656 11967 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:03.451346 11967 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:57:03.468924 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:03.453342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55262 (local address 127.11.175.193:46105)
0619 14:57:03.453593 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:03.453600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:03.453741 (+ 141us) server_negotiation.cc:407] Connection header received
0619 14:57:03.453986 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:03.453992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:03.454066 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:03.454171 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:03.455356 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.456361 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.457680 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.458276 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.458427 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:03.462485 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:03.462499 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:03.462515 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:03.462553 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:03.465523 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.466373 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.466381 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.466383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.466530 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.467044 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.467049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.467051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.467437 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:03.467753 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:03.468138 (+ 385us) server_negotiation.cc:299] Negotiation successful
0619 14:57:03.468419 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260619 14:57:03.475795 18077 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:03.473635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.11.175.193:46105)
0619 14:57:03.473829 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:03.473837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:03.474201 (+ 364us) server_negotiation.cc:407] Connection header received
0619 14:57:03.474397 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:03.474402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:03.474496 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:03.474603 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:03.475546 (+ 943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55272: BlockingRecv error: recv got EOF from 127.0.0.1:55272 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 14:57:03.476266 11967 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5cc' of type FILE
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:03.477931 11967 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:03.479488 11967 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:57:03.494921 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:03.480887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.11.175.193:46105)
0619 14:57:03.481111 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:03.481118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:03.481369 (+ 251us) server_negotiation.cc:407] Connection header received
0619 14:57:03.481544 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:03.481548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:03.481612 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:03.481695 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:03.482613 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.483367 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.484563 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.485003 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.485108 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:03.489068 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:03.489074 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:03.489077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:03.489140 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:03.491922 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.492754 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.492759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.492761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.492899 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.493383 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.493387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.493389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.493685 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:03.493928 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:03.494285 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 14:57:03.494555 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260619 14:57:03.499797 18077 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:03.498009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.11.175.193:46105)
0619 14:57:03.498264 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:03.498271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:03.498480 (+ 209us) server_negotiation.cc:407] Connection header received
0619 14:57:03.498612 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:03.498615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:03.498749 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:03.498837 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:03.499550 (+ 713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55284: BlockingRecv error: recv got EOF from 127.0.0.1:55284 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260619 14:57:03.499933 11967 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781880972997616-11967-0/krb5kdc/krb5cc' of type FILE
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:03.501688 11967 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:03.503137 11967 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:57:03.518955 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:03.504624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55298 (local address 127.11.175.193:46105)
0619 14:57:03.504883 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:03.504890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:03.505138 (+ 248us) server_negotiation.cc:407] Connection header received
0619 14:57:03.505229 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:03.505242 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:03.505314 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:03.505403 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:03.506338 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.507211 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.508469 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.508882 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.508990 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:03.512923 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:03.512929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:03.512933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:03.512973 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:03.515985 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.516762 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.516766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.516769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.516941 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.517357 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.517360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.517362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.517700 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:03.517918 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:03.518258 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 14:57:03.518497 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260619 14:57:03.520780 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 17673
I20260619 14:57:03.534341 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:03.521524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55300 (local address 127.11.175.193:46105)
0619 14:57:03.521692 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:03.521697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:03.522104 (+ 407us) server_negotiation.cc:407] Connection header received
0619 14:57:03.522303 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:03.522308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:03.522378 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:03.522471 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:03.523562 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.524265 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.525861 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.526301 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.526409 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:03.528737 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:03.528742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:03.528746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:03.528783 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:03.531680 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.532392 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.532397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.532399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.532556 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.532964 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.532969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.532971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.533270 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:03.533508 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:03.533872 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 14:57:03.534059 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 14:57:03.535073 17796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:03.626559 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 17673
I20260619 14:57:03.644361 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 17807
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:57:03.670161 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:03.645436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33192 (local address 127.11.175.194:46225)
0619 14:57:03.646147 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:03.646154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:03.646181 (+ 27us) server_negotiation.cc:407] Connection header received
0619 14:57:03.646250 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:03.646257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:03.646484 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:03.646680 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:03.647809 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.649010 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.650534 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.651515 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.651633 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:03.662824 (+ 11191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:03.662847 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:03.662864 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:03.662912 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:03.666294 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.667452 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.667460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.667463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.667638 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.668231 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.668235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.668238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.668670 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:03.668927 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:03.669566 (+ 639us) server_negotiation.cc:299] Negotiation successful
0619 14:57:03.669809 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":335,"threads_started":1}
I20260619 14:57:03.671654 17930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:03.758262 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 17807
I20260619 14:57:03.775218 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 17941
Jun 19 14:57:03 dist-test-slave-76rk krb5kdc[17567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:57:03.793371 18092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:03.776281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49960 (local address 127.11.175.195:33177)
0619 14:57:03.776774 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:03.776781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:03.776852 (+ 71us) server_negotiation.cc:407] Connection header received
0619 14:57:03.776915 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:03.776921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:03.777129 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:03.777340 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:03.778502 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.779619 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.780967 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:03.781576 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:03.781732 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:03.786417 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:03.786433 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:03.786450 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:03.786505 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:03.789872 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.790928 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.790936 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.790938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.791094 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:03.791751 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:03.791755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:03.791757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:03.792191 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:03.792450 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:03.792800 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 14:57:03.793070 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":294,"threads_started":1}
I20260619 14:57:03.795068 18064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:03.879840 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 17941
I20260619 14:57:03.896674 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 17580
I20260619 14:57:03.898109 17642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:04.006486 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 17580
2026-06-19T14:57:04Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4255 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:57:04 dist-test-slave-76rk krb5kdc[18097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:57:04 dist-test-slave-76rk krb5kdc[18097](info): set up 2 sockets
Jun 19 14:57:04 dist-test-slave-76rk krb5kdc[18097](info): commencing operation
krb5kdc: starting...
W20260619 14:57:05.135571 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:57:05 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:57:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:57:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.254.keytab.
Entry for principal oryx/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.254.keytab.
I20260619 14:57:05.376247 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:45535
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.254.keytab
--principal=oryx/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:45535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:05.720242 18110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:05.720629 18110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:05.720809 18110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:05.728287 18110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:05.728461 18110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:05.728554 18110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:05.728665 18110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:05.728756 18110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:05.738924 18110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:45535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.254.keytab
--principal=oryx/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:45535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:05.740381 18110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:05.741758 18110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:05.754957 18115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:05.754905 18118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:05.754988 18116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:05.755980 18110 server_base.cc:1034] running on GCE node
I20260619 14:57:05.756845 18110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:05.758469 18110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:05.759719 18110 hybrid_clock.cc:648] HybridClock initialized: now 1781881025759657 us; error 58 us; skew 500 ppm
I20260619 14:57:05.760102 18110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:05 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881025, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:05.763765 18110 init.cc:377] Logged in from keytab as oryx/127.11.175.254@KRBTEST.COM (short username oryx)
I20260619 14:57:05.765512 18110 webserver.cc:460] Webserver started at http://127.11.175.254:39087/ using document root <none> and password file <none>
I20260619 14:57:05.766325 18110 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:05.766498 18110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:05.766901 18110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:05.770380 18110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "e70967bec8d64eb780c54c87c293f1e4"
format_stamp: "Formatted at 2026-06-19 14:57:05 on dist-test-slave-76rk"
server_key: "bbbd5061e20721c61bb7cce157cb3158"
server_key_iv: "3161b8a59162edc81064df4164f6918f"
server_key_version: "encryptionkey@0"
I20260619 14:57:05.771343 18110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "e70967bec8d64eb780c54c87c293f1e4"
format_stamp: "Formatted at 2026-06-19 14:57:05 on dist-test-slave-76rk"
server_key: "bbbd5061e20721c61bb7cce157cb3158"
server_key_iv: "3161b8a59162edc81064df4164f6918f"
server_key_version: "encryptionkey@0"
I20260619 14:57:05.777387 18110 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 14:57:05.781574 18125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:05.783377 18110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:57:05.783591 18110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "e70967bec8d64eb780c54c87c293f1e4"
format_stamp: "Formatted at 2026-06-19 14:57:05 on dist-test-slave-76rk"
server_key: "bbbd5061e20721c61bb7cce157cb3158"
server_key_iv: "3161b8a59162edc81064df4164f6918f"
server_key_version: "encryptionkey@0"
I20260619 14:57:05.783852 18110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:05.797567 18110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:05.801862 18110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:05.802177 18110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:05.822324 18110 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:45535
I20260619 14:57:05.822366 18176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:45535 every 8 connection(s)
I20260619 14:57:05.823868 18110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:57:05.827973 18177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:05.829355 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 18110
I20260619 14:57:05.829578 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:57:05.830281 11967 external_mini_cluster.cc:1428] Setting key 91977a4bc82d0bec319de6cb7de11b72
I20260619 14:57:05.843231 18177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4: Bootstrap starting.
Jun 19 14:57:05 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:05.846401 18177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:05.847790 18177 log.cc:824] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:05.851389 18177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4: No bootstrap required, opened a new log
I20260619 14:57:05.855687 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:05.832643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.11.175.254:45535)
0619 14:57:05.833554 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:05.833573 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:05.833630 (+ 57us) server_negotiation.cc:407] Connection header received
0619 14:57:05.835508 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:05.835550 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:05.836151 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:05.836647 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:05.838051 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:05.840166 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:05.841711 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:05.842882 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:05.843076 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:05.847145 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:05.847156 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:05.847175 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:05.847224 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:05.850987 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:05.851996 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:05.852003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:05.852010 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:05.852205 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:05.852646 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:05.852649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:05.852651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:05.853232 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:05.853500 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:05.854008 (+ 508us) server_negotiation.cc:299] Negotiation successful
0619 14:57:05.854465 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":374,"threads_started":1}
I20260619 14:57:05.860741 18177 raft_consensus.cc:348] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } }
I20260619 14:57:05.861253 18177 raft_consensus.cc:374] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:05.861362 18177 raft_consensus.cc:729] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70967bec8d64eb780c54c87c293f1e4, State: Initialized, Role: FOLLOWER
I20260619 14:57:05.862013 18177 consensus_queue.cc:260] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } }
I20260619 14:57:05.862249 18177 raft_consensus.cc:388] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:57:05.862381 18177 raft_consensus.cc:482] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:57:05.862572 18177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:05.864813 18177 raft_consensus.cc:504] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } }
I20260619 14:57:05.865396 18177 leader_election.cc:302] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e70967bec8d64eb780c54c87c293f1e4; no voters:
I20260619 14:57:05.866058 18177 leader_election.cc:288] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:57:05.866230 18182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:05.867013 18182 raft_consensus.cc:686] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 1 LEADER]: Becoming Leader. State: Replica: e70967bec8d64eb780c54c87c293f1e4, State: Running, Role: LEADER
I20260619 14:57:05.867583 18182 consensus_queue.cc:237] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } }
I20260619 14:57:05.868312 18177 sys_catalog.cc:564] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:57:05.870518 18183 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } }
I20260619 14:57:05.870992 18183 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:05.874922 18184 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e70967bec8d64eb780c54c87c293f1e4. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } }
I20260619 14:57:05.875243 18184 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:05.875837 18189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:57:05.881939 18189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:57:05.893985 18189 catalog_manager.cc:1347] Generated new cluster ID: 0e8d9b9cd3b241228c5ff51005f028bc
I20260619 14:57:05.894115 18189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:57:05.915676 18189 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:57:05.917059 18189 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:57:05.927739 18189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4: Generated new TSK 0
I20260619 14:57:05.928644 18189 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab.
Entry for principal oryx/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab.
I20260619 14:57:06.002888 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45535
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab
--principal=oryx/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:06.366401 18203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:06.366816 18203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:06.366938 18203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:06.374568 18203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:06.374737 18203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:06.374869 18203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:57:06.384902 18203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab
--principal=oryx/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:06.386202 18203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:06.387622 18203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:06.401016 18208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:06.401036 18209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:06.401295 18211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:06.402444 18203 server_base.cc:1034] running on GCE node
I20260619 14:57:06.403208 18203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:06.404194 18203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:06.405409 18203 hybrid_clock.cc:648] HybridClock initialized: now 1781881026405362 us; error 34 us; skew 500 ppm
I20260619 14:57:06.405714 18203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:06 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881026, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:06.409201 18203 init.cc:377] Logged in from keytab as oryx/127.11.175.193@KRBTEST.COM (short username oryx)
I20260619 14:57:06.410773 18203 webserver.cc:460] Webserver started at http://127.11.175.193:36023/ using document root <none> and password file <none>
I20260619 14:57:06.411484 18203 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:06.411588 18203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:06.411840 18203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:06.415102 18203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "8d8f273611624092b7905bfbb3d2855b"
format_stamp: "Formatted at 2026-06-19 14:57:06 on dist-test-slave-76rk"
server_key: "c17a2277a9bab50068d788f4aca6170d"
server_key_iv: "7369549cf021086ed9bcb9ece60bd477"
server_key_version: "encryptionkey@0"
I20260619 14:57:06.415931 18203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "8d8f273611624092b7905bfbb3d2855b"
format_stamp: "Formatted at 2026-06-19 14:57:06 on dist-test-slave-76rk"
server_key: "c17a2277a9bab50068d788f4aca6170d"
server_key_iv: "7369549cf021086ed9bcb9ece60bd477"
server_key_version: "encryptionkey@0"
I20260619 14:57:06.421603 18203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 14:57:06.425299 18218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:06.426843 18203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:57:06.427022 18203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "8d8f273611624092b7905bfbb3d2855b"
format_stamp: "Formatted at 2026-06-19 14:57:06 on dist-test-slave-76rk"
server_key: "c17a2277a9bab50068d788f4aca6170d"
server_key_iv: "7369549cf021086ed9bcb9ece60bd477"
server_key_version: "encryptionkey@0"
I20260619 14:57:06.427196 18203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:06.442909 18203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:06.447500 18203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:06.447743 18203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:06.448781 18203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:06.450064 18203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:06.450147 18203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:06.450232 18203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:06.450316 18203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:06.487221 18203 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:39773
I20260619 14:57:06.487268 18330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:39773 every 8 connection(s)
I20260619 14:57:06.488646 18203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:57:06.489225 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 18203
I20260619 14:57:06.489440 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:57:06.490358 11967 external_mini_cluster.cc:1428] Setting key eb50085d83909f2a42fda2de868c3d27
Jun 19 14:57:06 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881026, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:06.509651 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:06.491610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:49603 (local address 127.11.175.254:45535)
0619 14:57:06.492091 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:06.492098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:06.493191 (+ 1093us) server_negotiation.cc:407] Connection header received
0619 14:57:06.495171 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:06.495177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:06.495296 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:06.495433 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:06.497000 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:06.497722 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:06.499416 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:06.499804 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:06.499942 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:06.503459 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:06.503464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:06.503468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:06.503500 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:06.506266 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:06.507208 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:06.507213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:06.507215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:06.507348 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:06.507817 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:06.507827 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:06.507829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:06.508108 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:06.508268 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:06.509039 (+ 771us) server_negotiation.cc:299] Negotiation successful
0619 14:57:06.509279 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":237,"threads_started":1}
WARNING: no policy specified for oryx/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 14:57:06.510807 18334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:06.492262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.193:49603)
0619 14:57:06.493037 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:06.493075 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:06.494885 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:06.495707 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:06.495716 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:06.496253 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:06.496818 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:06.496834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:06.497933 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:06.497937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:06.499238 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:06.499250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:06.500040 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:06.500049 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:06.500279 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:06.500963 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:06.500986 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:06.503224 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:06.506450 (+ 3226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:06.506461 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:06.506499 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:06.507065 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:06.507465 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:06.507470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:06.507474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:06.507692 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:06.508240 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:06.508249 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:06.508627 (+ 378us) client_negotiation.cc:769] Sending connection context
0619 14:57:06.508912 (+ 285us) client_negotiation.cc:240] Negotiation successful
0619 14:57:06.509504 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":338,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.194@KRBTEST.COM" created.
I20260619 14:57:06.512494 18331 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45535
I20260619 14:57:06.512846 18331 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:06.513669 18331 heartbeater.cc:511] Master 127.11.175.254:45535 requested a full tablet report, sending...
I20260619 14:57:06.516135 18142 ts_manager.cc:194] Registered new tserver with Master: 8d8f273611624092b7905bfbb3d2855b (127.11.175.193:39773)
I20260619 14:57:06.518432 18142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:49603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab.
Entry for principal oryx/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab.
I20260619 14:57:06.541757 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45535
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab
--principal=oryx/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:06.875751 18337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:06.876093 18337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:06.876263 18337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:06.883961 18337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:06.884128 18337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:06.884301 18337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:57:06.894879 18337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab
--principal=oryx/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:06.896375 18337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:06.897740 18337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:06.911758 18343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:06.911952 18342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:06.912873 18345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:06.913897 18337 server_base.cc:1034] running on GCE node
I20260619 14:57:06.914543 18337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:06.915637 18337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:06.916882 18337 hybrid_clock.cc:648] HybridClock initialized: now 1781881026916817 us; error 65 us; skew 500 ppm
I20260619 14:57:06.917227 18337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:06 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881026, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:06.920933 18337 init.cc:377] Logged in from keytab as oryx/127.11.175.194@KRBTEST.COM (short username oryx)
I20260619 14:57:06.922603 18337 webserver.cc:460] Webserver started at http://127.11.175.194:40283/ using document root <none> and password file <none>
I20260619 14:57:06.923590 18337 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:06.923755 18337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:06.924101 18337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:06.927484 18337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "c4ca402f97c0461f950e5f721b66d574"
format_stamp: "Formatted at 2026-06-19 14:57:06 on dist-test-slave-76rk"
server_key: "95b6717d370e90cc02f99fbc8f9389cb"
server_key_iv: "3f42cf89f212337ccfb2e42e3253cf16"
server_key_version: "encryptionkey@0"
I20260619 14:57:06.928378 18337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "c4ca402f97c0461f950e5f721b66d574"
format_stamp: "Formatted at 2026-06-19 14:57:06 on dist-test-slave-76rk"
server_key: "95b6717d370e90cc02f99fbc8f9389cb"
server_key_iv: "3f42cf89f212337ccfb2e42e3253cf16"
server_key_version: "encryptionkey@0"
I20260619 14:57:06.934152 18337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 14:57:06.938443 18352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:06.940387 18337 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 14:57:06.940667 18337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "c4ca402f97c0461f950e5f721b66d574"
format_stamp: "Formatted at 2026-06-19 14:57:06 on dist-test-slave-76rk"
server_key: "95b6717d370e90cc02f99fbc8f9389cb"
server_key_iv: "3f42cf89f212337ccfb2e42e3253cf16"
server_key_version: "encryptionkey@0"
I20260619 14:57:06.940922 18337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:06.962198 18337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:06.967247 18337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:06.967607 18337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:06.968786 18337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:06.970392 18337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:06.970498 18337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:06.970685 18337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:06.970772 18337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260619 14:57:07.007131 18337 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:39947
I20260619 14:57:07.007189 18464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:39947 every 8 connection(s)
I20260619 14:57:07.008476 18337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:57:07.014480 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 18337
I20260619 14:57:07.014688 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:57:07.015156 11967 external_mini_cluster.cc:1428] Setting key bf9c5b571d24bae628d3b596a5b9a3e1
Jun 19 14:57:07 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881026, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:07.030580 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:07.011299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:33753 (local address 127.11.175.254:45535)
0619 14:57:07.011799 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:07.011804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:07.012825 (+ 1021us) server_negotiation.cc:407] Connection header received
0619 14:57:07.014832 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:07.014838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:07.014936 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:07.015058 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:07.016771 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:07.017628 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:07.019340 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:07.019779 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:07.019921 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:07.024137 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:07.024143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:07.024147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:07.024188 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:07.027268 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:07.028104 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:07.028114 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:07.028117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:07.028247 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:07.028723 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:07.028727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:07.028729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:07.029025 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:07.029187 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:07.029935 (+ 748us) server_negotiation.cc:299] Negotiation successful
0619 14:57:07.030173 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":268,"threads_started":1}
I20260619 14:57:07.031731 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:07.011889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.194:33753)
0619 14:57:07.012654 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:07.012695 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:07.014455 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:07.015370 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:07.015380 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:07.015963 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:07.016583 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:07.016602 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:07.017788 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:07.017795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:07.019172 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:07.019184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:07.020022 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:07.020029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:07.020281 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:07.021140 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:07.021169 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:07.023840 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:07.027450 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:07.027471 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:07.027487 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:07.027953 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:07.028370 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:07.028377 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:07.028386 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:07.028581 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:07.029162 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:07.029172 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:07.029552 (+ 380us) client_negotiation.cc:769] Sending connection context
0619 14:57:07.029843 (+ 291us) client_negotiation.cc:240] Negotiation successful
0619 14:57:07.030439 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":288,"threads_started":1}
I20260619 14:57:07.033669 18465 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45535
I20260619 14:57:07.034080 18465 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:07.035190 18465 heartbeater.cc:511] Master 127.11.175.254:45535 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.11.175.195@KRBTEST.COM; defaulting to no policy
I20260619 14:57:07.037228 18142 ts_manager.cc:194] Registered new tserver with Master: c4ca402f97c0461f950e5f721b66d574 (127.11.175.194:39947)
I20260619 14:57:07.038264 18142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:33753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab.
Entry for principal oryx/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab.
I20260619 14:57:07.065817 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45535
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab
--principal=oryx/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:07.403533 18471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:07.403872 18471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:07.404043 18471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:07.411633 18471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:07.411806 18471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:07.412002 18471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:57:07.423064 18471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab
--principal=oryx/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:07.424506 18471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:07.425925 18471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:07.439611 18476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:07.439652 18477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:07.439716 18479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:07.440886 18471 server_base.cc:1034] running on GCE node
I20260619 14:57:07.441571 18471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:07.442595 18471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:07.443858 18471 hybrid_clock.cc:648] HybridClock initialized: now 1781881027443792 us; error 62 us; skew 500 ppm
I20260619 14:57:07.444209 18471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:07 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881027, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:07.447770 18471 init.cc:377] Logged in from keytab as oryx/127.11.175.195@KRBTEST.COM (short username oryx)
I20260619 14:57:07.449384 18471 webserver.cc:460] Webserver started at http://127.11.175.195:38385/ using document root <none> and password file <none>
I20260619 14:57:07.450112 18471 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:07.450259 18471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:07.450572 18471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:07.453747 18471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "23a1f3ae85a54085b8628e1b77ffc780"
format_stamp: "Formatted at 2026-06-19 14:57:07 on dist-test-slave-76rk"
server_key: "53acd6c101c6261224539f49501fe0a9"
server_key_iv: "e4bc8b626b2aaed7fb1fca2857ec0906"
server_key_version: "encryptionkey@0"
I20260619 14:57:07.454613 18471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "23a1f3ae85a54085b8628e1b77ffc780"
format_stamp: "Formatted at 2026-06-19 14:57:07 on dist-test-slave-76rk"
server_key: "53acd6c101c6261224539f49501fe0a9"
server_key_iv: "e4bc8b626b2aaed7fb1fca2857ec0906"
server_key_version: "encryptionkey@0"
I20260619 14:57:07.460513 18471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 14:57:07.464428 18486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:07.466048 18471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 14:57:07.466260 18471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "23a1f3ae85a54085b8628e1b77ffc780"
format_stamp: "Formatted at 2026-06-19 14:57:07 on dist-test-slave-76rk"
server_key: "53acd6c101c6261224539f49501fe0a9"
server_key_iv: "e4bc8b626b2aaed7fb1fca2857ec0906"
server_key_version: "encryptionkey@0"
I20260619 14:57:07.466488 18471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:07.493593 18471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:07.498517 18471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:07.498862 18471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:07.499883 18471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:07.501575 18471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:07.501693 18471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:07.501852 18471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:07.501927 18471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:07.522795 18331 heartbeater.cc:503] Master 127.11.175.254:45535 was elected leader, sending a full tablet report...
I20260619 14:57:07.537575 18471 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:44397
I20260619 14:57:07.537664 18598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:44397 every 8 connection(s)
I20260619 14:57:07.538959 18471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:57:07.539587 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 18471
I20260619 14:57:07.539800 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:57:07.540450 11967 external_mini_cluster.cc:1428] Setting key 7986fceb2bec0c380e79b5637a35ca83
Jun 19 14:57:07 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881027, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:07.561822 18601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:07.541869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:52659 (local address 127.11.175.254:45535)
0619 14:57:07.542321 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:07.542327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:07.543760 (+ 1433us) server_negotiation.cc:407] Connection header received
0619 14:57:07.545737 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:07.545741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:07.545824 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:07.545921 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:07.547851 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:07.548616 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:07.550391 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:07.550844 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:07.550955 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:07.555171 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:07.555182 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:07.555186 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:07.555238 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:07.558423 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:07.559402 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:07.559406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:07.559408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:07.559563 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:07.560111 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:07.560114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:07.560116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:07.560393 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:07.560533 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:07.561348 (+ 815us) server_negotiation.cc:299] Negotiation successful
0619 14:57:07.561544 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":443,"threads_started":1}
I20260619 14:57:07.563017 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:07.542547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.195:52659)
0619 14:57:07.543429 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:07.543643 (+ 214us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:07.545462 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:07.546219 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:07.546229 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:07.547020 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:07.547653 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:07.547673 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:07.548774 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:07.548778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:07.550206 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:07.550220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:07.551058 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:07.551064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:07.551292 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:07.552028 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:07.552052 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:07.554893 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:07.558629 (+ 3736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:07.558699 (+ 70us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:07.558723 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:07.559264 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:07.559700 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:07.559704 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:07.559707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:07.559979 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:07.560551 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:07.560557 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:07.560989 (+ 432us) client_negotiation.cc:769] Sending connection context
0619 14:57:07.561247 (+ 258us) client_negotiation.cc:240] Negotiation successful
0619 14:57:07.561669 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":291,"threads_started":1}
I20260619 14:57:07.564615 18599 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45535
I20260619 14:57:07.564971 18599 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:07.565761 18599 heartbeater.cc:511] Master 127.11.175.254:45535 requested a full tablet report, sending...
I20260619 14:57:07.567531 18142 ts_manager.cc:194] Registered new tserver with Master: 23a1f3ae85a54085b8628e1b77ffc780 (127.11.175.195:44397)
I20260619 14:57:07.568634 18142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:52659
I20260619 14:57:07.569646 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.175.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.253.keytab.
Entry for principal oryx/127.11.175.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.253.keytab.
I20260619 14:57:07.620657 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.253:36139
--webserver_interface=127.11.175.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.253.keytab
--principal=oryx/127.11.175.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:45535,127.11.175.253:36139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:07.966945 18605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:07.967221 18605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:07.967308 18605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:07.974881 18605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:07.974989 18605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:07.975036 18605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:07.975093 18605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:07.975137 18605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:07.985297 18605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:45535,127.11.175.253:36139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.253.keytab
--principal=oryx/127.11.175.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.253:36139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.175.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
W20260619 14:57:07.985616 18605 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.175.254:45535,127.11.175.253:36139'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 14:57:07.986603 18605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:07.988124 18605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:08.000021 18613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:08.000069 18610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:08.000231 18611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:08.001096 18605 server_base.cc:1034] running on GCE node
I20260619 14:57:08.001837 18605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:08.003321 18605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:08.004575 18605 hybrid_clock.cc:648] HybridClock initialized: now 1781881028004504 us; error 50 us; skew 500 ppm
I20260619 14:57:08.004905 18605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881028, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:08.008713 18605 init.cc:377] Logged in from keytab as oryx/127.11.175.253@KRBTEST.COM (short username oryx)
I20260619 14:57:08.012724 18605 webserver.cc:460] Webserver started at http://127.11.175.253:44907/ using document root <none> and password file <none>
I20260619 14:57:08.013460 18605 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:08.013559 18605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:08.013877 18605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:08.016986 18605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data/instance:
uuid: "70fba17797984460af4e7336695f1cec"
format_stamp: "Formatted at 2026-06-19 14:57:08 on dist-test-slave-76rk"
server_key: "f407c446783ad448523be872366f6cbb"
server_key_iv: "1226ada15902580fa3201ee7324fb51f"
server_key_version: "encryptionkey@0"
I20260619 14:57:08.017822 18605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal/instance:
uuid: "70fba17797984460af4e7336695f1cec"
format_stamp: "Formatted at 2026-06-19 14:57:08 on dist-test-slave-76rk"
server_key: "f407c446783ad448523be872366f6cbb"
server_key_iv: "1226ada15902580fa3201ee7324fb51f"
server_key_version: "encryptionkey@0"
I20260619 14:57:08.023296 18605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 14:57:08.027822 18620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:08.029307 18605 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 14:57:08.029528 18605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal
uuid: "70fba17797984460af4e7336695f1cec"
format_stamp: "Formatted at 2026-06-19 14:57:08 on dist-test-slave-76rk"
server_key: "f407c446783ad448523be872366f6cbb"
server_key_iv: "1226ada15902580fa3201ee7324fb51f"
server_key_version: "encryptionkey@0"
I20260619 14:57:08.029758 18605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:08.041536 18465 heartbeater.cc:503] Master 127.11.175.254:45535 was elected leader, sending a full tablet report...
I20260619 14:57:08.046435 18605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:08.050714 18605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:08.050978 18605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:08.069921 18605 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.253:36139
I20260619 14:57:08.069901 18671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.253:36139 every 8 connection(s)
I20260619 14:57:08.071475 18605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
I20260619 14:57:08.073899 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 18605
I20260619 14:57:08.075708 18672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:08.085474 18672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.253@KRBTEST.COM
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881028, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.253@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:08.100308 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.076199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51324 (local address 127.11.175.253:36139)
0619 14:57:08.077036 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.077051 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.077100 (+ 49us) server_negotiation.cc:407] Connection header received
0619 14:57:08.079261 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.079291 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.079850 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.080348 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.082246 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.084315 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.085586 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.086727 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.086903 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.092086 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.092093 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.092103 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.092139 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.095606 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.096532 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.096540 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.096548 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.096754 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.097145 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.097150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.097154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.097832 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.098064 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.098690 (+ 626us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.099125 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":375,"threads_started":1}
I20260619 14:57:08.105655 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.088585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48282 (local address 127.11.175.254:45535)
0619 14:57:08.089235 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.089241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.089819 (+ 578us) server_negotiation.cc:407] Connection header received
0619 14:57:08.090056 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.090060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.090126 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.090214 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.091468 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.092418 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.094703 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.095281 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.095407 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.099261 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.099266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.099270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.099308 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.102678 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.103572 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.103577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.103581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.103713 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.104125 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.104129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.104131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.104446 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.104610 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.105056 (+ 446us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.105283 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":299,"threads_started":1}
I20260619 14:57:08.105692 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.088588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:48282)
0619 14:57:08.089665 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.089722 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.089891 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.090342 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.090353 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.091065 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.091307 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.091316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.092577 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.092584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.094450 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.094466 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.095494 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.095503 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.095707 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.096334 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.096351 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.099063 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.102813 (+ 3750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.102821 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.102826 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.103432 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.103831 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.103836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.103840 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.103992 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.104580 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.104587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.104759 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.104942 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.105276 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":583,"threads_started":1}
I20260619 14:57:08.107504 18672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881028, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.253@KRBTEST.COM for oryx/127.11.175.253@KRBTEST.COM
I20260619 14:57:08.120054 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.108239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:36139 (local address 127.0.0.1:51338)
0619 14:57:08.108439 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.108460 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.108562 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.108836 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.108840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.109226 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.109353 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.109361 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.110323 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.110330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.111256 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.111267 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.111923 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.111928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.112144 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.112716 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.112732 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.114724 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.117604 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.117610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.117614 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.117970 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.118339 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.118346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.118349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.118481 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.119050 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.119055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.119201 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.119336 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.119630 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":4}
I20260619 14:57:08.120132 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.108391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51338 (local address 127.11.175.253:36139)
0619 14:57:08.108557 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.108562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.108579 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:57:08.108668 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.108674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.108726 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.108824 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.109478 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.110187 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.111391 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.111783 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.111912 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.114891 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.114895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.114898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.114925 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.117454 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.118089 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.118097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.118100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.118235 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.118581 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.118588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.118592 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.118933 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.119087 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.119407 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.119774 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":206,"server-negotiator.queue_time_us":47}
I20260619 14:57:08.125093 18672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: Bootstrap starting.
I20260619 14:57:08.128193 18672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:08.129176 18672 log.cc:824] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:08.131985 18672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: No bootstrap required, opened a new log
I20260619 14:57:08.139114 18672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } }
I20260619 14:57:08.139442 18672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:08.139539 18672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70fba17797984460af4e7336695f1cec, State: Initialized, Role: FOLLOWER
I20260619 14:57:08.140120 18672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } }
I20260619 14:57:08.140925 18679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } } }
I20260619 14:57:08.141283 18679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:57:08.141973 18672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: configured and running, proceeding with master startup.
W20260619 14:57:08.151547 18690 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 14:57:08.151705 18690 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 14:57:08.154098 18605 master_runner.cc:428] Detected that this master 70fba17797984460af4e7336695f1cec is joining an existing cluster
I20260619 14:57:08.154210 18605 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.175.253:36139
I20260619 14:57:08.176584 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.158228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48298 (local address 127.11.175.254:45535)
0619 14:57:08.158616 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.158625 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.159828 (+ 1203us) server_negotiation.cc:407] Connection header received
0619 14:57:08.160056 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.160065 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.160220 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.160333 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.162257 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.163715 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.165815 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.166486 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.166594 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.169239 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.169246 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.169252 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.169295 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.173193 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.174062 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.174068 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.174072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.174222 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.174723 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.174728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.174732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.175139 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.175268 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.175949 (+ 681us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.176195 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260619 14:57:08.177733 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.158249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:48298)
0619 14:57:08.159715 (+ 1466us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.159733 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.159902 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.160572 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.160580 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.161483 (+ 903us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.162040 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.162057 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.163900 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.163908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.165630 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.165641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.166903 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.166910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.167134 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.167852 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.167874 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.169029 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.173365 (+ 4336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.173381 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.173385 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.173943 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.174344 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.174355 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.174361 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.174549 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.175365 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.175382 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.175610 (+ 228us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.176860 (+ 1250us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.177203 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"mutex_wait_us":1,"thread_start_us":1186,"threads_started":1}
I20260619 14:57:08.179013 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.158870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51342 (local address 127.11.175.253:36139)
0619 14:57:08.159087 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.159095 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.164053 (+ 4958us) server_negotiation.cc:407] Connection header received
0619 14:57:08.164281 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.164287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.164356 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.164420 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.165751 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.166799 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.168766 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.169542 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.169642 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.172071 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.172080 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.172086 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.172131 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.176019 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.176778 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.176783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.176785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.176941 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.177356 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.177360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.177363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.177712 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.177882 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.178382 (+ 500us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.178574 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":34}
I20260619 14:57:08.179908 18697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.158836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:36139 (local address 127.0.0.1:51342)
0619 14:57:08.163936 (+ 5100us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.163961 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.164144 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.164551 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.164557 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.165233 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.165601 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.165612 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.166990 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.166995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.168620 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.168633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.170007 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.170017 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.170215 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.170819 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.170842 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.171884 (+ 1042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.176147 (+ 4263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.176151 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.176155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.176660 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.177066 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.177071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.177079 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.177235 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.177986 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.177993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.178132 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.179172 (+ 1040us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.179468 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4961,"thread_start_us":4752,"threads_started":1}
WARNING: no policy specified for oryx/127.11.175.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.252@KRBTEST.COM" created.
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881025, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.254@KRBTEST.COM for oryx/127.11.175.253@KRBTEST.COM
I20260619 14:57:08.200361 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.184683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.11.175.253:36139)
0619 14:57:08.184862 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.184867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.185588 (+ 721us) server_negotiation.cc:407] Connection header received
0619 14:57:08.185958 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.185963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.186059 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.186172 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.187448 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.188300 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.189777 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.190256 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.190400 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.193711 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.193716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.193719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.193771 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.197446 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.198140 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.198147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.198151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.198351 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.198882 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.198885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.198887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.199149 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.199305 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.199776 (+ 471us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.199999 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 14:57:08.200424 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.184766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:36139 (local address 127.0.0.1:51358)
0619 14:57:08.185479 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.185513 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.185756 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.186221 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.186229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.187004 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.187293 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.187308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.188437 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.188443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.189614 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.189626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.190424 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.190430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.190689 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.191374 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.191392 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.193524 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.197590 (+ 4066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.197604 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.197611 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.198023 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.198537 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.198542 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.198546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.198762 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.199295 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.199304 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.199481 (+ 177us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.199660 (+ 179us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.200006 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":320,"threads_started":1}
I20260619 14:57:08.202489 18141 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.175.253:36139: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "70fba17797984460af4e7336695f1cec"
member_type: NON_VOTER
last_known_addr {
host: "127.11.175.253"
port: 36139
}
attrs {
promote: true
}
}
dest_uuid: "e70967bec8d64eb780c54c87c293f1e4"
cas_config_opid_index: -1
I20260619 14:57:08.203533 18141 consensus_queue.cc:237] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } }
I20260619 14:57:08.214416 18702 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 70fba17797984460af4e7336695f1cec (127.11.175.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } } }
I20260619 14:57:08.215703 18647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:08.220170 18647 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 FOLLOWER]: Refusing update from remote peer e70967bec8d64eb780c54c87c293f1e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 14:57:08.220224 18702 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.175.253:36139
I20260619 14:57:08.220700 18703 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } } }
I20260619 14:57:08.221042 18703 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:08.221055 18605 master.cc:537] Master@127.11.175.253:36139 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.252.keytab.
Entry for principal oryx/127.11.175.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.252.keytab.
I20260619 14:57:08.224942 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.252:36403
--webserver_interface=127.11.175.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.252.keytab
--principal=oryx/127.11.175.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:45535,127.11.175.253:36139,127.11.175.252:36403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:57:08.227118 18705 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 14:57:08.231608 18129 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 14:57:08.238390 18129 consensus_peers.cc:487] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 -> Peer 70fba17797984460af4e7336695f1cec (127.11.175.253:36139): Couldn't send request to peer 70fba17797984460af4e7336695f1cec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 14:57:08.240350 18605 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:08.240772 18605 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:08.240978 18605 tablet_replica.cc:330] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: stopping tablet replica
I20260619 14:57:08.257712 18605 master.cc:552] Master@127.11.175.253:36139 shutdown complete.
I20260619 14:57:08.257834 18605 master_runner.cc:305] Clearing existing system tablet
I20260619 14:57:08.258209 18605 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:08.258332 18605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:08.260658 18605 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260619 14:57:08.262260 18709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:08.262814 18605 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:08.263094 18605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 14:57:08.263178 18605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal
uuid: "70fba17797984460af4e7336695f1cec"
format_stamp: "Formatted at 2026-06-19 14:57:08 on dist-test-slave-76rk"
server_key: "f407c446783ad448523be872366f6cbb"
server_key_iv: "1226ada15902580fa3201ee7324fb51f"
server_key_version: "encryptionkey@0"
I20260619 14:57:08.266557 18605 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:08.271900 18605 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 14:57:08.272104 18605 log.cc:1196] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 14:57:08.272554 18605 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: Deleting consensus metadata
I20260619 14:57:08.272855 18605 master_runner.cc:315] Copying system tablet from 127.11.175.254:45535
I20260619 14:57:08.276221 18605 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: tablet copy: Beginning tablet copy session from remote peer at address 127.11.175.254:45535
I20260619 14:57:08.295648 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.277083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48302 (local address 127.11.175.254:45535)
0619 14:57:08.277306 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.277316 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.283253 (+ 5937us) server_negotiation.cc:407] Connection header received
0619 14:57:08.283490 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.283496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.283584 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.283710 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.284613 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.285372 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.286628 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.287224 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.287353 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.289569 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.289577 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.289585 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.289627 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.292813 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.293570 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.293576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.293580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.293710 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.294100 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.294106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.294109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.294399 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.294565 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.295002 (+ 437us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.295233 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 14:57:08.295744 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.277064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:48302)
0619 14:57:08.283097 (+ 6033us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.283119 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.283331 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.283734 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.283739 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.284216 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.284450 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.284459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.285531 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.285536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.286471 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.286482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.287371 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.287377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.287592 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.288413 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.288432 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.289261 (+ 829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.293012 (+ 3751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.293018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.293021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.293428 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.293829 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.293834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.293838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.293989 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.294524 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.294529 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.294753 (+ 224us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.294901 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.295246 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5873,"thread_start_us":5787,"threads_started":1}
I20260619 14:57:08.296916 18162 tablet_copy_service.cc:140] P e70967bec8d64eb780c54c87c293f1e4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 70fba17797984460af4e7336695f1cec ({username='oryx', principal='oryx/127.11.175.253@KRBTEST.COM'} at 127.0.0.1:48302)
I20260619 14:57:08.297215 18162 tablet_copy_service.cc:161] P e70967bec8d64eb780c54c87c293f1e4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 70fba17797984460af4e7336695f1cec at {username='oryx', principal='oryx/127.11.175.253@KRBTEST.COM'} at 127.0.0.1:48302: session id = 70fba17797984460af4e7336695f1cec-00000000000000000000000000000000
I20260619 14:57:08.302117 18162 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 14:57:08.304744 18605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 14:57:08.311543 18605 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: tablet copy: Starting download of 0 data blocks...
I20260619 14:57:08.311882 18605 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: tablet copy: Starting download of 1 WAL segments...
I20260619 14:57:08.314800 18605 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 14:57:08.319343 18162 tablet_copy_service.cc:342] P e70967bec8d64eb780c54c87c293f1e4: Request end of tablet copy session 70fba17797984460af4e7336695f1cec-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.175.253@KRBTEST.COM'} at 127.0.0.1:48302
I20260619 14:57:08.319584 18162 tablet_copy_service.cc:433] P e70967bec8d64eb780c54c87c293f1e4: ending tablet copy session 70fba17797984460af4e7336695f1cec-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 70fba17797984460af4e7336695f1cec
I20260619 14:57:08.331529 18605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:08.334188 18716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:08.340137 18717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:08.348412 18605 server_base.cc:1034] running on GCE node
W20260619 14:57:08.348771 18719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:08.349261 18605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:08.350047 18605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:08.351310 18605 hybrid_clock.cc:648] HybridClock initialized: now 1781881028351242 us; error 34 us; skew 500 ppm
I20260619 14:57:08.351631 18605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881028, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:08.353951 18605 init.cc:377] Logged in from keytab as oryx/127.11.175.253@KRBTEST.COM (short username oryx)
I20260619 14:57:08.355561 18605 webserver.cc:460] Webserver started at http://127.11.175.253:43241/ using document root <none> and password file <none>
I20260619 14:57:08.355986 18605 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:08.356119 18605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:08.359344 18605 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 14:57:08.361408 18726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:08.361842 18605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260619 14:57:08.362031 18605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal
uuid: "70fba17797984460af4e7336695f1cec"
format_stamp: "Formatted at 2026-06-19 14:57:08 on dist-test-slave-76rk"
server_key: "f407c446783ad448523be872366f6cbb"
server_key_iv: "1226ada15902580fa3201ee7324fb51f"
server_key_version: "encryptionkey@0"
I20260619 14:57:08.362264 18605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:08.393502 18605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:08.398053 18605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:08.418887 18605 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.253:36139
I20260619 14:57:08.418931 18777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.253:36139 every 8 connection(s)
I20260619 14:57:08.420073 18605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
I20260619 14:57:08.425765 18778 sys_catalog.cc:263] Verifying existing consensus state
I20260619 14:57:08.429540 18778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: Bootstrap starting.
I20260619 14:57:08.447960 18778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 14:57:08.448526 18778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: Bootstrap complete.
I20260619 14:57:08.449591 18778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } }
I20260619 14:57:08.449841 18778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70fba17797984460af4e7336695f1cec, State: Initialized, Role: LEARNER
I20260619 14:57:08.450059 18778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } }
I20260619 14:57:08.451081 18781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } } }
I20260619 14:57:08.451292 18778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:57:08.451527 18781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: This master's current role is: LEARNER
I20260619 14:57:08.461294 18792 catalog_manager.cc:1259] Loaded cluster ID: 0e8d9b9cd3b241228c5ff51005f028bc
I20260619 14:57:08.461403 18792 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: loading cluster ID for follower catalog manager: success
I20260619 14:57:08.464859 18792 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: acquiring CA information for follower catalog manager: success
I20260619 14:57:08.467388 18792 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 14:57:08.571802 18599 heartbeater.cc:503] Master 127.11.175.254:45535 was elected leader, sending a full tablet report...
W20260619 14:57:08.625725 18707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:08.626008 18707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:08.626152 18707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:08.633881 18707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:08.633999 18707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:08.634079 18707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:08.634177 18707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:08.634240 18707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:08.644614 18707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:45535,127.11.175.253:36139,127.11.175.252:36403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.252.keytab
--principal=oryx/127.11.175.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.252:36403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.175.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:08.646045 18707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:08.647418 18707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:08.660264 18797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:08.660228 18798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:08.660192 18800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:08.661386 18707 server_base.cc:1034] running on GCE node
I20260619 14:57:08.662164 18707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:08.663830 18707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:08.665066 18707 hybrid_clock.cc:648] HybridClock initialized: now 1781881028665028 us; error 28 us; skew 500 ppm
I20260619 14:57:08.665417 18707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881028, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:08.672705 18707 init.cc:377] Logged in from keytab as oryx/127.11.175.252@KRBTEST.COM (short username oryx)
I20260619 14:57:08.674353 18707 webserver.cc:460] Webserver started at http://127.11.175.252:42381/ using document root <none> and password file <none>
I20260619 14:57:08.675140 18707 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:08.675288 18707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:08.675625 18707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:08.678694 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data/instance:
uuid: "706f093f7bd749bfa4a548ab8a7d8953"
format_stamp: "Formatted at 2026-06-19 14:57:08 on dist-test-slave-76rk"
server_key: "500388272662c3ac129741e44b7930e6"
server_key_iv: "efc91433b74678373909686b8f6dcbef"
server_key_version: "encryptionkey@0"
I20260619 14:57:08.679471 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal/instance:
uuid: "706f093f7bd749bfa4a548ab8a7d8953"
format_stamp: "Formatted at 2026-06-19 14:57:08 on dist-test-slave-76rk"
server_key: "500388272662c3ac129741e44b7930e6"
server_key_iv: "efc91433b74678373909686b8f6dcbef"
server_key_version: "encryptionkey@0"
I20260619 14:57:08.684773 18707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 14:57:08.688402 18807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:08.689967 18707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 14:57:08.690155 18707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal
uuid: "706f093f7bd749bfa4a548ab8a7d8953"
format_stamp: "Formatted at 2026-06-19 14:57:08 on dist-test-slave-76rk"
server_key: "500388272662c3ac129741e44b7930e6"
server_key_iv: "efc91433b74678373909686b8f6dcbef"
server_key_version: "encryptionkey@0"
I20260619 14:57:08.690325 18707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:08.715291 18707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:08.720860 18707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:08.721164 18707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:08.723080 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.715651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:36139 (local address 127.0.0.1:51362)
0619 14:57:08.716196 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.716217 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.716365 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.716892 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.716898 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.716927 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:08.717186 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.717204 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.718579 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.718586 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.720698 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.720714 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.722069 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.722079 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.722341 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.722401 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.722510 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.722672 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":282,"threads_started":1}
I20260619 14:57:08.723093 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.715904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51362 (local address 127.11.175.253:36139)
0619 14:57:08.716550 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.716555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.716572 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:57:08.716632 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.716636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.716706 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.716807 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:08.717388 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.718408 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.720859 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.721829 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.721982 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.722112 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.722550 (+ 438us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.722711 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":380,"threads_started":1}
I20260619 14:57:08.724496 18753 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260619 14:57:08.725239 18781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: SysCatalogTable state changed. Reason: New leader e70967bec8d64eb780c54c87c293f1e4. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } } }
I20260619 14:57:08.725528 18781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: This master's current role is: LEARNER
I20260619 14:57:08.743404 18707 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.252:36403
I20260619 14:57:08.743435 18860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.252:36403 every 8 connection(s)
I20260619 14:57:08.744913 18707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
I20260619 14:57:08.748826 18861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:08.751482 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 18707
I20260619 14:57:08.756880 18861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.252@KRBTEST.COM
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881028, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.252@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:08.775214 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.759686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48308 (local address 127.11.175.254:45535)
0619 14:57:08.759886 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.759892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.760676 (+ 784us) server_negotiation.cc:407] Connection header received
0619 14:57:08.760838 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.760844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.760920 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.761030 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.762189 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.763043 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.764569 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.764967 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.765100 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.768699 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.768705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.768710 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.768752 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.771964 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.772840 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.772846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.772849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.773054 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.773486 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.773490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.773493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.773827 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.773989 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.774571 (+ 582us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.774860 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 14:57:08.776325 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.753785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.11.175.252:36403)
0619 14:57:08.754744 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.754764 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.754835 (+ 71us) server_negotiation.cc:407] Connection header received
0619 14:57:08.756840 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.756867 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.757295 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.757705 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.759403 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.761198 (+ 1795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.762547 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.763352 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.763577 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.768368 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.768377 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.768389 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.768423 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.772003 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.772848 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.772856 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.772859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.773045 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.773462 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.773474 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.773477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.774143 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.774369 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.774820 (+ 451us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.775204 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":130,"server-negotiator.queue_time_us":679,"thread_start_us":395,"threads_started":1}
I20260619 14:57:08.776378 18865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.759581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:48308)
0619 14:57:08.760491 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.760567 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.760719 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.761091 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.761097 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.761763 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.762014 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.762023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.763176 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.763183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.764416 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.764429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.765196 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.765203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.765354 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.765997 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.766023 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.768519 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.772142 (+ 3623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.772153 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.772166 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.772711 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.773170 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.773174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.773178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.773381 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.773951 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.773957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.774151 (+ 194us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.774489 (+ 338us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.775052 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"spinlock_wait_cycles":17152,"thread_start_us":475,"threads_started":1}
I20260619 14:57:08.777853 18861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881028, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.252@KRBTEST.COM for oryx/127.11.175.253@KRBTEST.COM
I20260619 14:57:08.791028 18865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.778756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:36139 (local address 127.0.0.1:51372)
0619 14:57:08.778947 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.778967 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.779081 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.779420 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.779423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.779736 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.779939 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.779948 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.780903 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.780907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.781811 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.781822 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.782573 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.782577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.782782 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.783300 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.783317 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.785263 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.788453 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.788457 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.788459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.788956 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.789305 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.789309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.789311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.789436 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.790105 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.790110 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.790247 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.790378 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.790621 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 14:57:08.791082 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.778789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51372 (local address 127.11.175.253:36139)
0619 14:57:08.779089 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.779097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.779122 (+ 25us) server_negotiation.cc:407] Connection header received
0619 14:57:08.779199 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.779203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.779282 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.779376 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.780061 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.780773 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.781966 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.782421 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.782528 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.785463 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.785468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.785473 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.785506 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.788313 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.789074 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.789078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.789081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.789210 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.789560 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.789564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.789566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.789862 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.790013 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.790572 (+ 559us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.790831 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260619 14:57:08.792199 18861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881028, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.252@KRBTEST.COM for oryx/127.11.175.252@KRBTEST.COM
I20260619 14:57:08.805065 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.792944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.11.175.252:36403)
0619 14:57:08.793113 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.793117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.793132 (+ 15us) server_negotiation.cc:407] Connection header received
0619 14:57:08.793213 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.793216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.793268 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.793343 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.794117 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.794941 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.796087 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.796605 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.796702 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.799843 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.799848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.799851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.799882 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.802400 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.803130 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.803138 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.803141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.803258 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.803629 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.803636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.803639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.803909 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.804057 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.804407 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.804732 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":127,"server-negotiator.queue_time_us":84}
I20260619 14:57:08.805065 18865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.792850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:36403 (local address 127.0.0.1:46790)
0619 14:57:08.793019 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.793039 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.793122 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.793376 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.793379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.793810 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.793968 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.793976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.795055 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.795058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.795952 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.795962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.796728 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.796732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.796896 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.797523 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.797540 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.799641 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.802530 (+ 2889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.802546 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.802550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.803018 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.803361 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.803367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.803371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.803530 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.804018 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.804023 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.804162 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.804316 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.804598 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 14:57:08.810840 18861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Bootstrap starting.
I20260619 14:57:08.814559 18861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:08.815861 18861 log.cc:824] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:08.818845 18861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: No bootstrap required, opened a new log
I20260619 14:57:08.825941 18861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } }
I20260619 14:57:08.826325 18861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:08.826471 18861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 706f093f7bd749bfa4a548ab8a7d8953, State: Initialized, Role: FOLLOWER
I20260619 14:57:08.827073 18861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } }
I20260619 14:57:08.828061 18867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } } }
I20260619 14:57:08.828446 18867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:57:08.829219 18861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 14:57:08.840202 18878 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 14:57:08.840359 18878 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 14:57:08.844744 18707 master_runner.cc:428] Detected that this master 706f093f7bd749bfa4a548ab8a7d8953 is joining an existing cluster
I20260619 14:57:08.844857 18707 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.175.252:36403
I20260619 14:57:08.871624 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.851530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.11.175.252:36403)
0619 14:57:08.851676 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.851682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.854870 (+ 3188us) server_negotiation.cc:407] Connection header received
0619 14:57:08.855081 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.855087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.855175 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.855282 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.856801 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.858016 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.860020 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.860836 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.860974 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.864022 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.864031 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.864039 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.864079 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.868022 (+ 3943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.868980 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.868986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.868991 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.869171 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.869676 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.869683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.869688 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.870135 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.870282 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.870933 (+ 651us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.871191 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":38}
I20260619 14:57:08.872500 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.849052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51388 (local address 127.11.175.253:36139)
0619 14:57:08.849266 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.849273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.851521 (+ 2248us) server_negotiation.cc:407] Connection header received
0619 14:57:08.851765 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.851773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.851861 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.851940 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.854218 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.855351 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.858468 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.859583 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.859686 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.863411 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.863418 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.863422 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.863461 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.867778 (+ 4317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.868905 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.868910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.868914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.869123 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.869786 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.869791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.869795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.870215 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.870356 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.871182 (+ 826us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.871501 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 14:57:08.872737 18886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.851676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:36403 (local address 127.0.0.1:46796)
0619 14:57:08.854786 (+ 3110us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.854804 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.854945 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.855426 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.855431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.856198 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.856642 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.856656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.858233 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.858239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.859877 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.859890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.861220 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.861229 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.861565 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.862480 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.862514 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.863837 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.868168 (+ 4331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.868174 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.868178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.868861 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.869312 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.869317 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.869320 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.869562 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.870346 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.870350 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.870564 (+ 214us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.871896 (+ 1332us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.872175 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2908,"spinlock_wait_cycles":45952,"thread_start_us":2761,"threads_started":1}
I20260619 14:57:08.874554 18885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.850609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:36139 (local address 127.0.0.1:51388)
0619 14:57:08.851415 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.851433 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.851615 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.852174 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.852178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.853587 (+ 1409us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.854014 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.854033 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.855571 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.855577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.858300 (+ 2723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.858312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.860107 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.860118 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.861166 (+ 1048us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.861825 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.861852 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.863176 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.868003 (+ 4827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.868008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.868025 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.868741 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.869306 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.869313 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.869319 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.869650 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.870448 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.870453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.870794 (+ 341us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.873265 (+ 2471us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.873739 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":39,"thread_start_us":419,"threads_started":1}
I20260619 14:57:08.878218 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.849029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48320 (local address 127.11.175.254:45535)
0619 14:57:08.849212 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.849219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.850568 (+ 1349us) server_negotiation.cc:407] Connection header received
0619 14:57:08.850959 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.850967 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.851064 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.851147 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.853556 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.854790 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.856979 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.857402 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.857571 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.860070 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.860076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.860083 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.860130 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.871589 (+ 11459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.874555 (+ 2966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.874560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.874563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.874773 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.876390 (+ 1617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.876394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.876396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.876689 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.876806 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.877515 (+ 709us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.877892 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 14:57:08.878233 18884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.849126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:48320)
0619 14:57:08.850409 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.850431 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.850726 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.851380 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.851386 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.852383 (+ 997us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.853320 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.853344 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.854930 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.854938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.856806 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.856816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.857665 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.857672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.857839 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.858567 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.858596 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.859850 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.873815 (+ 13965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.873819 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.873822 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.874364 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.876030 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.876035 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.876039 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.876241 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.876874 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.876879 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.877108 (+ 229us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.877320 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.877724 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1127,"mutex_wait_us":4,"thread_start_us":555,"threads_started":1}
I20260619 14:57:08.882946 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 18203
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881025, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.254@KRBTEST.COM for oryx/127.11.175.252@KRBTEST.COM
Jun 19 14:57:08 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.193@KRBTEST.COM
I20260619 14:57:08.901670 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.885227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46800 (local address 127.11.175.252:36403)
0619 14:57:08.885389 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.885396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.885418 (+ 22us) server_negotiation.cc:407] Connection header received
0619 14:57:08.885539 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.885544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.885640 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.885737 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.886862 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.887903 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.889745 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.890462 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.890618 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.894746 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.894752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.894756 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.894792 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.898506 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.899441 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.899449 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.899453 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.899586 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.900037 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.900043 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.900046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.900409 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.900568 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.901107 (+ 539us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.901330 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 14:57:08.901675 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.884926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:36403 (local address 127.0.0.1:46800)
0619 14:57:08.885121 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.885141 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:08.885249 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:08.885943 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:08.885961 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:08.886457 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:08.886689 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.886703 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.888039 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.888047 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.889573 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:08.889593 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.890687 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.890694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:08.890900 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.891644 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:08.891673 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:08.894386 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:08.898696 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.898703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.898707 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.899297 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.899717 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:08.899723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:08.899727 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:08.899897 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:08.900576 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:08.900583 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:08.900769 (+ 186us) client_negotiation.cc:769] Sending connection context
0619 14:57:08.900943 (+ 174us) client_negotiation.cc:240] Negotiation successful
0619 14:57:08.901262 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260619 14:57:08.903671 18141 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.175.252:36403: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953"
member_type: NON_VOTER
last_known_addr {
host: "127.11.175.252"
port: 36403
}
attrs {
promote: true
}
}
dest_uuid: "e70967bec8d64eb780c54c87c293f1e4"
cas_config_opid_index: 5
I20260619 14:57:08.904564 18141 consensus_queue.cc:237] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } }
I20260619 14:57:08.907337 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.884120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.11.175.193:39773)
0619 14:57:08.884871 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:08.884878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:08.884904 (+ 26us) server_negotiation.cc:407] Connection header received
0619 14:57:08.884986 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:08.884997 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:08.885320 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:08.885532 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:08.887019 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.888708 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.890455 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:08.891533 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:08.891695 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:08.897978 (+ 6283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:08.897996 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:08.898016 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:08.898061 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:08.902376 (+ 4315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.903435 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.903448 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.903454 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.903644 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:08.904361 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:08.904367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:08.904370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:08.905011 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:08.905327 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:08.906584 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0619 14:57:08.906909 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":390,"threads_started":1}
I20260619 14:57:08.907011 18703 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 706f093f7bd749bfa4a548ab8a7d8953 (127.11.175.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:08.912478 18326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:08.914487 18753 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 LEARNER]: Refusing update from remote peer e70967bec8d64eb780c54c87c293f1e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 14:57:08.916283 18836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:08.925136 18703 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.175.252:36403
I20260619 14:57:08.926069 18707 master.cc:537] Master@127.11.175.252:36403 shutting down...
I20260619 14:57:08.926899 18889 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 14:57:08.927217 18836 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 FOLLOWER]: Refusing update from remote peer e70967bec8d64eb780c54c87c293f1e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 14:57:08.957456 18892 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:08.959862 18753 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 706f093f7bd749bfa4a548ab8a7d8953 (127.11.175.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:08.969545 18707 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:08.970156 18707 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:08.970412 18707 tablet_replica.cc:330] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: stopping tablet replica
W20260619 14:57:08.971796 18129 consensus_peers.cc:487] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 -> Peer 706f093f7bd749bfa4a548ab8a7d8953 (127.11.175.252:36403): Couldn't send request to peer 706f093f7bd749bfa4a548ab8a7d8953. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 14:57:08.974188 18889 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:08.974581 18889 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:08.979970 18707 master.cc:552] Master@127.11.175.252:36403 shutdown complete.
I20260619 14:57:08.980165 18707 master_runner.cc:305] Clearing existing system tablet
I20260619 14:57:08.980778 18707 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:08.981009 18707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:08.982163 18781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: NON_VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: true } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:08.982522 18781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: This master's current role is: LEARNER
I20260619 14:57:08.984266 18703 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4: attempting to promote NON_VOTER 70fba17797984460af4e7336695f1cec to VOTER
I20260619 14:57:08.985546 18703 consensus_queue.cc:237] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } }
I20260619 14:57:08.989496 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.988807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:36403 (local address 127.0.0.1:46804)
0619 14:57:08.989038 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.989229 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:36403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 14:57:08.990770 18752 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 LEARNER]: Refusing update from remote peer e70967bec8d64eb780c54c87c293f1e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 14:57:08.991737 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:08.991219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:36403 (local address 127.0.0.1:46820)
0619 14:57:08.991421 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:08.991530 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:36403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 14:57:08.992050 18129 consensus_peers.cc:487] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 -> Peer 706f093f7bd749bfa4a548ab8a7d8953 (127.11.175.252:36403): Couldn't send request to peer 706f093f7bd749bfa4a548ab8a7d8953. Status: Network error: Client connection negotiation failed: client connection to 127.11.175.252:36403: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 14:57:08.992743 18892 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 14:57:08.997991 18892 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 70fba17797984460af4e7336695f1cec (127.11.175.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:08.999364 18707 fs_manager.cc:709] Time spent opening directory manager: real 0.017s user 0.005s sys 0.011s
I20260619 14:57:09.003473 18752 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 70fba17797984460af4e7336695f1cec (127.11.175.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:09.007428 18897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:09.007876 18707 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:09.008345 18707 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.007s sys 0.000s
I20260619 14:57:09.008523 18707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal
uuid: "706f093f7bd749bfa4a548ab8a7d8953"
format_stamp: "Formatted at 2026-06-19 14:57:08 on dist-test-slave-76rk"
server_key: "500388272662c3ac129741e44b7930e6"
server_key_iv: "efc91433b74678373909686b8f6dcbef"
server_key_version: "encryptionkey@0"
I20260619 14:57:09.009132 18781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:09.009405 18781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:57:09.014901 18707 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:09.019412 18703 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:09.019723 18703 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:09.020184 18892 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:09.020452 18892 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:09.022529 18707 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 14:57:09.022838 18707 log.cc:1196] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 14:57:09.023487 18707 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Deleting consensus metadata
I20260619 14:57:09.023770 18707 master_runner.cc:315] Copying system tablet from 127.11.175.254:45535
I20260619 14:57:09.030448 18707 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: tablet copy: Beginning tablet copy session from remote peer at address 127.11.175.254:45535
I20260619 14:57:09.044112 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.031327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48322 (local address 127.11.175.254:45535)
0619 14:57:09.031657 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:09.031667 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:09.032218 (+ 551us) server_negotiation.cc:407] Connection header received
0619 14:57:09.032499 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:09.032506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:09.032596 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:09.032706 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:09.033586 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.034324 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.035668 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.036104 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.036238 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.038246 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:09.038253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:09.038258 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:09.038300 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:09.041096 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:09.041875 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:09.041881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:09.041885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:09.042061 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:09.042455 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:09.042462 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:09.042465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:09.042838 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:09.043021 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:09.043427 (+ 406us) server_negotiation.cc:299] Negotiation successful
0619 14:57:09.043662 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260619 14:57:09.045406 18903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.031283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:48322)
0619 14:57:09.032095 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:09.032116 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:09.032347 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:09.032743 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:09.032749 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:09.033226 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:09.033425 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:09.033434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.034475 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.034488 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:09.035506 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:09.035518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.036273 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.036279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:09.036462 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.037175 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:09.037203 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:09.038047 (+ 844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:09.041251 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:09.041256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:09.041260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:09.041736 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:09.042177 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:09.042182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:09.042186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:09.042337 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:09.042959 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:09.042964 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:09.043182 (+ 218us) client_negotiation.cc:769] Sending connection context
0619 14:57:09.043333 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 14:57:09.043677 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":558,"threads_started":1}
I20260619 14:57:09.046003 18162 tablet_copy_service.cc:140] P e70967bec8d64eb780c54c87c293f1e4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 706f093f7bd749bfa4a548ab8a7d8953 ({username='oryx', principal='oryx/127.11.175.252@KRBTEST.COM'} at 127.0.0.1:48322)
I20260619 14:57:09.046177 18162 tablet_copy_service.cc:161] P e70967bec8d64eb780c54c87c293f1e4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 706f093f7bd749bfa4a548ab8a7d8953 at {username='oryx', principal='oryx/127.11.175.252@KRBTEST.COM'} at 127.0.0.1:48322: session id = 706f093f7bd749bfa4a548ab8a7d8953-00000000000000000000000000000000
I20260619 14:57:09.048534 18162 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 14:57:09.050220 18707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 14:57:09.055778 18707 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: tablet copy: Starting download of 0 data blocks...
I20260619 14:57:09.056102 18707 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: tablet copy: Starting download of 1 WAL segments...
I20260619 14:57:09.058589 18707 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 14:57:09.062050 18162 tablet_copy_service.cc:342] P e70967bec8d64eb780c54c87c293f1e4: Request end of tablet copy session 706f093f7bd749bfa4a548ab8a7d8953-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.175.252@KRBTEST.COM'} at 127.0.0.1:48322
I20260619 14:57:09.062199 18162 tablet_copy_service.cc:433] P e70967bec8d64eb780c54c87c293f1e4: ending tablet copy session 706f093f7bd749bfa4a548ab8a7d8953-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 706f093f7bd749bfa4a548ab8a7d8953
I20260619 14:57:09.073076 18707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:09.075806 18904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:09.080415 18905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:09.088521 18907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:09.088828 18707 server_base.cc:1034] running on GCE node
I20260619 14:57:09.089346 18707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:09.090138 18707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:09.091398 18707 hybrid_clock.cc:648] HybridClock initialized: now 1781881029091326 us; error 53 us; skew 500 ppm
I20260619 14:57:09.091682 18707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:09 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881029, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:09.093673 18707 init.cc:377] Logged in from keytab as oryx/127.11.175.252@KRBTEST.COM (short username oryx)
I20260619 14:57:09.099769 18707 webserver.cc:460] Webserver started at http://127.11.175.252:46565/ using document root <none> and password file <none>
I20260619 14:57:09.100129 18707 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:09.100243 18707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:09.102803 18707 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260619 14:57:09.104628 18914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:09.105109 18707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260619 14:57:09.105249 18707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal
uuid: "706f093f7bd749bfa4a548ab8a7d8953"
format_stamp: "Formatted at 2026-06-19 14:57:08 on dist-test-slave-76rk"
server_key: "500388272662c3ac129741e44b7930e6"
server_key_iv: "efc91433b74678373909686b8f6dcbef"
server_key_version: "encryptionkey@0"
I20260619 14:57:09.105463 18707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:09.114058 18707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:09.117628 18707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:09.125918 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 18203
I20260619 14:57:09.155608 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:39773
--local_ip_for_outbound_sockets=127.11.175.193
--tserver_master_addrs=127.11.175.254:45535
--webserver_port=36023
--webserver_interface=127.11.175.193
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab
--principal=oryx/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.175.254:45535,127.11.175.253:36139
--tserver_master_addrs=127.11.175.254:45535,127.11.175.253:36139,127.11.175.252:36403 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:57:09.164892 18707 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.252:36403
I20260619 14:57:09.164938 18965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.252:36403 every 8 connection(s)
I20260619 14:57:09.166112 18707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
I20260619 14:57:09.171231 18967 sys_catalog.cc:263] Verifying existing consensus state
I20260619 14:57:09.175091 18967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Bootstrap starting.
I20260619 14:57:09.194958 18967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 14:57:09.195477 18967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Bootstrap complete.
I20260619 14:57:09.196502 18967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } }
I20260619 14:57:09.196792 18967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 706f093f7bd749bfa4a548ab8a7d8953, State: Initialized, Role: LEARNER
I20260619 14:57:09.197427 18967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } }
I20260619 14:57:09.198513 18967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:57:09.198251 18970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:09.198798 18970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: This master's current role is: LEARNER
I20260619 14:57:09.206192 18981 catalog_manager.cc:1259] Loaded cluster ID: 0e8d9b9cd3b241228c5ff51005f028bc
I20260619 14:57:09.206307 18981 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: loading cluster ID for follower catalog manager: success
I20260619 14:57:09.209795 18981 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: acquiring CA information for follower catalog manager: success
I20260619 14:57:09.212404 18981 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 14:57:09.496193 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.488270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:36403 (local address 127.0.0.1:46832)
0619 14:57:09.488496 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:09.488522 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:09.488698 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:09.489581 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:09.489585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:09.489610 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:09.489782 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:09.489790 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.491728 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.491734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:09.493708 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:09.493742 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.495258 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.495265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:09.495522 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.495576 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 14:57:09.495699 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 14:57:09.495863 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 14:57:09.496243 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.488468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46832 (local address 127.11.175.252:36403)
0619 14:57:09.489121 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:09.489129 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:09.489152 (+ 23us) server_negotiation.cc:407] Connection header received
0619 14:57:09.489237 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:09.489243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:09.489348 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:09.489525 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:09.489932 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.491568 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.493907 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.495059 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.495258 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.495487 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:09.495727 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 14:57:09.495871 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":295,"threads_started":1}
I20260619 14:57:09.497406 18941 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 14:57:09.498044 18970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e70967bec8d64eb780c54c87c293f1e4. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:09.498386 18970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: This master's current role is: LEARNER
W20260619 14:57:09.530202 18966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:09.530493 18966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:09.530674 18966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:09.538211 18966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:09.538370 18966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:09.538546 18966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:57:09.548743 18966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab
--principal=oryx/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:39773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=36023
--tserver_master_addrs=127.11.175.254:45535,127.11.175.253:36139,127.11.175.252:36403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:09.550097 18966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:09.551543 18966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:09.565636 18988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:09.565659 18990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:09.566053 18987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:09.567042 18966 server_base.cc:1034] running on GCE node
I20260619 14:57:09.567692 18966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:09.568756 18966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:09.569952 18966 hybrid_clock.cc:648] HybridClock initialized: now 1781881029569908 us; error 33 us; skew 500 ppm
I20260619 14:57:09.570314 18966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:09 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881029, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:09.573755 18966 init.cc:377] Logged in from keytab as oryx/127.11.175.193@KRBTEST.COM (short username oryx)
I20260619 14:57:09.575691 18966 webserver.cc:460] Webserver started at http://127.11.175.193:36023/ using document root <none> and password file <none>
I20260619 14:57:09.576547 18966 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:09.576671 18966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:09.582329 18966 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 14:57:09.585266 18997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:09.586828 18966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 14:57:09.586977 18966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "8d8f273611624092b7905bfbb3d2855b"
format_stamp: "Formatted at 2026-06-19 14:57:06 on dist-test-slave-76rk"
server_key: "c17a2277a9bab50068d788f4aca6170d"
server_key_iv: "7369549cf021086ed9bcb9ece60bd477"
server_key_version: "encryptionkey@0"
I20260619 14:57:09.588153 18966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:09.613997 18966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:09.618777 18966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:09.619053 18966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:09.620121 18966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:09.621636 18966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:09.621726 18966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:09.621799 18966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:09.621855 18966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:09.658993 18966 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:39773
I20260619 14:57:09.659089 19109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:39773 every 8 connection(s)
I20260619 14:57:09.660532 18966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:57:09.669912 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 18966
I20260619 14:57:09.670287 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 18337
Jun 19 14:57:09 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881029, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
Jun 19 14:57:09 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881029, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.253@KRBTEST.COM
Jun 19 14:57:09 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881029, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.252@KRBTEST.COM
Jun 19 14:57:09 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.194@KRBTEST.COM
I20260619 14:57:09.689222 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.663615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:52319 (local address 127.11.175.252:36403)
0619 14:57:09.663825 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:09.663831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:09.665403 (+ 1572us) server_negotiation.cc:407] Connection header received
0619 14:57:09.667805 (+ 2402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:09.667811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:09.667895 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:09.667987 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:09.670190 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.671369 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.674455 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.675172 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.675274 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.682031 (+ 6757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:09.682041 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:09.682049 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:09.682101 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:09.685779 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:09.686731 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:09.686737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:09.686743 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:09.686914 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:09.687451 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:09.687457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:09.687460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:09.687823 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:09.687952 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:09.688601 (+ 649us) server_negotiation.cc:299] Negotiation successful
0619 14:57:09.688858 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 14:57:09.689814 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.663862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:51031 (local address 127.11.175.254:45535)
0619 14:57:09.664424 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:09.664432 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:09.665400 (+ 968us) server_negotiation.cc:407] Connection header received
0619 14:57:09.667694 (+ 2294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:09.667700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:09.667787 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:09.667925 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:09.670229 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.671366 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.674420 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.675095 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.675191 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.680732 (+ 5541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:09.680738 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:09.680741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:09.680776 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:09.684469 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:09.685836 (+ 1367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:09.685845 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:09.685852 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:09.686009 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:09.686591 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:09.686596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:09.686602 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:09.687059 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:09.687211 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:09.688569 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0619 14:57:09.688846 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":227,"threads_started":1}
I20260619 14:57:09.691506 19115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.664108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.193:51031)
0619 14:57:09.665226 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:09.665275 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:09.667424 (+ 2149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:09.668177 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:09.668185 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:09.668978 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:09.670005 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:09.670022 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.671630 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.671635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:09.674256 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:09.674264 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.675540 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.675564 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:09.676015 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.676987 (+ 972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:09.677021 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:09.680447 (+ 3426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:09.684927 (+ 4480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:09.684936 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:09.684954 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:09.685677 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:09.686171 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:09.686178 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:09.686183 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:09.686450 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:09.687407 (+ 957us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:09.687416 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:09.687984 (+ 568us) client_negotiation.cc:769] Sending connection context
0619 14:57:09.690260 (+ 2276us) client_negotiation.cc:240] Negotiation successful
0619 14:57:09.690604 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":280,"spinlock_wait_cycles":6784,"thread_start_us":543,"threads_started":1}
I20260619 14:57:09.691531 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.664108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:36403 (local address 127.11.175.193:52319)
0619 14:57:09.665256 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:09.665276 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:09.667448 (+ 2172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:09.668181 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:09.668187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:09.669251 (+ 1064us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:09.669924 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:09.669953 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.671593 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.671598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:09.674266 (+ 2668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:09.674281 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.675605 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.675613 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:09.675993 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.677190 (+ 1197us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:09.677211 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:09.681784 (+ 4573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:09.685946 (+ 4162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:09.685953 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:09.685958 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:09.686544 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:09.687077 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:09.687082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:09.687086 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:09.687309 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:09.688055 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:09.688060 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:09.688231 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 14:57:09.689482 (+ 1251us) client_negotiation.cc:240] Negotiation successful
0619 14:57:09.689984 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"mutex_wait_us":485,"spinlock_wait_cycles":85248,"thread_start_us":600,"threads_started":1}
I20260619 14:57:09.692034 19118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.666304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:34721 (local address 127.11.175.253:36139)
0619 14:57:09.666826 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:09.666830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:09.666851 (+ 21us) server_negotiation.cc:407] Connection header received
0619 14:57:09.667802 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:09.667807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:09.667895 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:09.667986 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:09.670302 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.671594 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.674427 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.675087 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.675214 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.681187 (+ 5973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:09.681225 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:09.681233 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:09.681312 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:09.686602 (+ 5290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:09.687585 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:09.687590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:09.687593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:09.687723 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:09.688265 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:09.688271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:09.688276 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:09.688750 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:09.688876 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:09.691384 (+ 2508us) server_negotiation.cc:299] Negotiation successful
0619 14:57:09.691658 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":273,"threads_started":1}
I20260619 14:57:09.692425 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.665854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:36139 (local address 127.11.175.193:34721)
0619 14:57:09.666434 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:09.666449 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:09.667448 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:09.668213 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:09.668217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:09.669251 (+ 1034us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:09.670120 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:09.670135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.671760 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.671771 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:09.674248 (+ 2477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:09.674263 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.675560 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.675567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:09.676029 (+ 462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.677030 (+ 1001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:09.677047 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:09.680949 (+ 3902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:09.686785 (+ 5836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:09.686792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:09.686796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:09.687426 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:09.687870 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:09.687876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:09.687879 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:09.688133 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:09.688950 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:09.688955 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:09.689100 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 14:57:09.689280 (+ 180us) client_negotiation.cc:240] Negotiation successful
0619 14:57:09.689893 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":308,"spinlock_wait_cycles":22016,"thread_start_us":306,"threads_started":1}
I20260619 14:57:09.693565 19111 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45535
I20260619 14:57:09.693624 19110 heartbeater.cc:347] Connected to a master server at 127.11.175.252:36403
I20260619 14:57:09.693794 19112 heartbeater.cc:347] Connected to a master server at 127.11.175.253:36139
I20260619 14:57:09.693956 19110 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:09.694905 19110 heartbeater.cc:511] Master 127.11.175.252:36403 requested a full tablet report, sending...
I20260619 14:57:09.694938 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:09.671717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51020 (local address 127.11.175.194:39947)
0619 14:57:09.672849 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:09.672855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:09.672877 (+ 22us) server_negotiation.cc:407] Connection header received
0619 14:57:09.672959 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:09.672968 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:09.673206 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:09.675453 (+ 2247us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:09.675740 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.677290 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.679318 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:09.680109 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:09.682807 (+ 2698us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:09.686217 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:09.686232 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:09.686251 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:09.686294 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:09.690313 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:09.691273 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:09.691281 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:09.691284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:09.691449 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:09.692751 (+ 1302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:09.692756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:09.692759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:09.693244 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:09.693625 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:09.694264 (+ 639us) server_negotiation.cc:299] Negotiation successful
0619 14:57:09.694566 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":990,"thread_start_us":829,"threads_started":1}
I20260619 14:57:09.693953 19112 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:09.693953 19111 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:09.696167 19112 heartbeater.cc:511] Master 127.11.175.253:36139 requested a full tablet report, sending...
I20260619 14:57:09.696345 18460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:09.696333 19111 heartbeater.cc:511] Master 127.11.175.254:45535 requested a full tablet report, sending...
I20260619 14:57:09.697700 18141 ts_manager.cc:194] Re-registered known tserver with Master: 8d8f273611624092b7905bfbb3d2855b (127.11.175.193:39773)
I20260619 14:57:09.697743 18743 ts_manager.cc:194] Registered new tserver with Master: 8d8f273611624092b7905bfbb3d2855b (127.11.175.193:39773)
I20260619 14:57:09.699265 18141 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:51031
I20260619 14:57:09.699297 18931 ts_manager.cc:194] Registered new tserver with Master: 8d8f273611624092b7905bfbb3d2855b (127.11.175.193:39773)
I20260619 14:57:09.784444 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 18337
I20260619 14:57:09.802282 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:39947
--local_ip_for_outbound_sockets=127.11.175.194
--tserver_master_addrs=127.11.175.254:45535
--webserver_port=40283
--webserver_interface=127.11.175.194
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab
--principal=oryx/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.175.254:45535,127.11.175.253:36139
--tserver_master_addrs=127.11.175.254:45535,127.11.175.253:36139,127.11.175.252:36403 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:57:09.904357 18889 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: NON_VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: true } } }
I20260619 14:57:09.904824 18889 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:09.906821 18889 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4: attempting to promote NON_VOTER 706f093f7bd749bfa4a548ab8a7d8953 to VOTER
I20260619 14:57:09.907481 18889 consensus_queue.cc:237] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } }
I20260619 14:57:09.909583 18753 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 FOLLOWER]: Refusing update from remote peer e70967bec8d64eb780c54c87c293f1e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260619 14:57:09.909821 18941 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 LEARNER]: Refusing update from remote peer e70967bec8d64eb780c54c87c293f1e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260619 14:57:09.910387 19124 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260619 14:57:09.910800 18889 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260619 14:57:09.913546 19124 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 706f093f7bd749bfa4a548ab8a7d8953 (127.11.175.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:09.914328 18753 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 706f093f7bd749bfa4a548ab8a7d8953 (127.11.175.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:09.917798 19124 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:09.918222 19124 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:09.918623 19124 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:09.918895 19124 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:09.921643 19127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:09.922412 19127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:57:09.923822 18889 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:09.924109 18889 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:09.924535 18941 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 706f093f7bd749bfa4a548ab8a7d8953 (127.11.175.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:09.927683 18970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e70967bec8d64eb780c54c87c293f1e4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:09.927947 18970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 14:57:10.162416 19122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:10.162757 19122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:10.162873 19122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:10.170810 19122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:10.170971 19122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:10.171152 19122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:57:10.181624 19122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab
--principal=oryx/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:39947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=40283
--tserver_master_addrs=127.11.175.254:45535,127.11.175.253:36139,127.11.175.252:36403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:10.183104 19122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:10.184423 19122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:10.198679 19132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:10.198628 19135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:10.198608 19133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:10.199950 19122 server_base.cc:1034] running on GCE node
I20260619 14:57:10.200644 19122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:10.201843 19122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:10.203102 19122 hybrid_clock.cc:648] HybridClock initialized: now 1781881030203050 us; error 73 us; skew 500 ppm
I20260619 14:57:10.203431 19122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:10 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881030, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:10.206935 19122 init.cc:377] Logged in from keytab as oryx/127.11.175.194@KRBTEST.COM (short username oryx)
I20260619 14:57:10.208339 19122 webserver.cc:460] Webserver started at http://127.11.175.194:40283/ using document root <none> and password file <none>
I20260619 14:57:10.208989 19122 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:10.209105 19122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:10.214594 19122 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 14:57:10.217365 19142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:10.219002 19122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:57:10.219146 19122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "c4ca402f97c0461f950e5f721b66d574"
format_stamp: "Formatted at 2026-06-19 14:57:06 on dist-test-slave-76rk"
server_key: "95b6717d370e90cc02f99fbc8f9389cb"
server_key_iv: "3f42cf89f212337ccfb2e42e3253cf16"
server_key_version: "encryptionkey@0"
I20260619 14:57:10.220078 19122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:10.251922 19122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:10.256922 19122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:10.257181 19122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:10.258327 19122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:10.259964 19122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:10.260059 19122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:10.260192 19122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:10.260293 19122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:10.296638 19122 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:39947
I20260619 14:57:10.296752 19254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:39947 every 8 connection(s)
I20260619 14:57:10.298153 19122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:57:10.307184 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 19122
I20260619 14:57:10.307564 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 18471
Jun 19 14:57:10 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881030, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.253@KRBTEST.COM
Jun 19 14:57:10 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881030, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.252@KRBTEST.COM
Jun 19 14:57:10 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881030, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
Jun 19 14:57:10 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.195@KRBTEST.COM
I20260619 14:57:10.351645 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.301524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:50551 (local address 127.11.175.254:45535)
0619 14:57:10.330955 (+ 29431us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:10.330962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:10.330993 (+ 31us) server_negotiation.cc:407] Connection header received
0619 14:57:10.331054 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:10.331059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:10.331134 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:10.332519 (+ 1385us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:10.336023 (+ 3504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.337128 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.338862 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.339522 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.339664 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.343717 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:10.343724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:10.343729 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:10.343771 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:10.348050 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.348808 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.348815 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.348818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.348991 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.349393 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.349398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.349402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.349759 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:10.349891 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:10.350926 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0619 14:57:10.351197 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29288,"thread_start_us":29990,"threads_started":1}
I20260619 14:57:10.352120 19260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.301525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:42159 (local address 127.11.175.253:36139)
0619 14:57:10.331624 (+ 30099us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:10.331631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:10.331649 (+ 18us) server_negotiation.cc:407] Connection header received
0619 14:57:10.331707 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:10.331715 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:10.331792 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:10.331902 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:10.334648 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.335780 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.337907 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.338536 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.338676 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.343331 (+ 4655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:10.343338 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:10.343343 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:10.343403 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:10.347347 (+ 3944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.348426 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.348432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.348437 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.348597 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.349188 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.349194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.349197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.349606 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:10.349751 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:10.351199 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0619 14:57:10.351462 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29969,"thread_start_us":29831,"threads_started":1}
I20260619 14:57:10.353060 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.302266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.194:50551)
0619 14:57:10.323606 (+ 21340us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:10.323630 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:10.323782 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:10.332114 (+ 8332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:10.332118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:10.333031 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:10.334367 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.334379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.337266 (+ 2887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.337273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.338502 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.338513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.340166 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.340176 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.340383 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.341036 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:10.341057 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:10.343467 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:10.348198 (+ 4731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:10.348204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:10.348208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:10.348701 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:10.349108 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:10.349117 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:10.349120 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:10.349272 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:10.349856 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:10.349862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:10.350437 (+ 575us) client_negotiation.cc:769] Sending connection context
0619 14:57:10.350808 (+ 371us) client_negotiation.cc:240] Negotiation successful
0619 14:57:10.351445 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21168,"mutex_wait_us":1143,"spinlock_wait_cycles":59904,"thread_start_us":20397,"threads_started":1}
I20260619 14:57:10.353341 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.302266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:36139 (local address 127.11.175.194:42159)
0619 14:57:10.318843 (+ 16577us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:10.318935 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:10.322587 (+ 3652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:10.331994 (+ 9407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:10.332007 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:10.333058 (+ 1051us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:10.333884 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.333905 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.336006 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.336012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.337748 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.337761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.338914 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.338921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.339200 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.340082 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:10.340116 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:10.343022 (+ 2906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:10.347581 (+ 4559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:10.347592 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:10.347615 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:10.348283 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:10.348741 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:10.348748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:10.348753 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:10.349039 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:10.349853 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:10.349862 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:10.350438 (+ 576us) client_negotiation.cc:769] Sending connection context
0619 14:57:10.350808 (+ 370us) client_negotiation.cc:240] Negotiation successful
0619 14:57:10.352844 (+ 2036us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16056,"mutex_wait_us":84,"spinlock_wait_cycles":6912,"thread_start_us":20564,"threads_started":1}
I20260619 14:57:10.353407 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.309462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54174 (local address 127.11.175.195:44397)
0619 14:57:10.324752 (+ 15290us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:10.324763 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:10.325815 (+ 1052us) server_negotiation.cc:407] Connection header received
0619 14:57:10.325925 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:10.325939 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:10.326325 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:10.326541 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:10.329727 (+ 3186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.331464 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.333282 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.334213 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.334373 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.345170 (+ 10797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:10.345185 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:10.345200 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:10.345241 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:10.349708 (+ 4467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.350517 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.350525 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.350527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.350727 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.351238 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.351243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.351246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.351670 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:10.351975 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:10.352579 (+ 604us) server_negotiation.cc:299] Negotiation successful
0619 14:57:10.352851 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15092,"thread_start_us":14913,"threads_started":1}
I20260619 14:57:10.354861 19257 heartbeater.cc:347] Connected to a master server at 127.11.175.253:36139
I20260619 14:57:10.355078 18594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:10.355233 19257 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:10.356114 19257 heartbeater.cc:511] Master 127.11.175.253:36139 requested a full tablet report, sending...
I20260619 14:57:10.356927 19256 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45535
I20260619 14:57:10.357060 19256 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:10.357544 19256 heartbeater.cc:511] Master 127.11.175.254:45535 requested a full tablet report, sending...
I20260619 14:57:10.358182 18743 ts_manager.cc:194] Registered new tserver with Master: c4ca402f97c0461f950e5f721b66d574 (127.11.175.194:39947)
I20260619 14:57:10.358566 18141 ts_manager.cc:194] Re-registered known tserver with Master: c4ca402f97c0461f950e5f721b66d574 (127.11.175.194:39947)
I20260619 14:57:10.359429 19266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.328133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:36403 (local address 127.11.175.194:40201)
0619 14:57:10.330713 (+ 2580us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:10.330748 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:10.330922 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:10.331992 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:10.332005 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:10.333501 (+ 1496us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:10.333890 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.333912 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.335381 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.335387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.337592 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.337606 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.338768 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.338777 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.339195 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.340161 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:10.340194 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:10.343917 (+ 3723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:10.354073 (+ 10156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:10.354081 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:10.354088 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:10.356241 (+ 2153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:10.356783 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:10.356788 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:10.356791 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:10.356993 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:10.357999 (+ 1006us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:10.358007 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:10.358217 (+ 210us) client_negotiation.cc:769] Sending connection context
0619 14:57:10.358422 (+ 205us) client_negotiation.cc:240] Negotiation successful
0619 14:57:10.359124 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2388,"mutex_wait_us":65,"thread_start_us":2187,"threads_started":1}
I20260619 14:57:10.359694 19265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.324638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:40201 (local address 127.11.175.252:36403)
0619 14:57:10.328608 (+ 3970us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:10.328617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:10.331071 (+ 2454us) server_negotiation.cc:407] Connection header received
0619 14:57:10.331160 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:10.331168 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:10.331273 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:10.331385 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:10.334107 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.335223 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.337770 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.338468 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.338621 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.344162 (+ 5541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:10.344172 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:10.344177 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:10.344229 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:10.349793 (+ 5564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.356390 (+ 6597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.356401 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.356407 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.356635 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.357158 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.357163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.357167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.357835 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:10.358806 (+ 971us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:10.358970 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 14:57:10.359298 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3793,"thread_start_us":3690,"threads_started":1}
I20260619 14:57:10.359793 18141 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:50551
I20260619 14:57:10.360806 19255 heartbeater.cc:347] Connected to a master server at 127.11.175.252:36403
I20260619 14:57:10.360944 19255 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:10.361392 19255 heartbeater.cc:511] Master 127.11.175.252:36403 requested a full tablet report, sending...
I20260619 14:57:10.363142 18931 ts_manager.cc:194] Registered new tserver with Master: c4ca402f97c0461f950e5f721b66d574 (127.11.175.194:39947)
I20260619 14:57:10.447983 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 18471
I20260619 14:57:10.465082 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:44397
--local_ip_for_outbound_sockets=127.11.175.195
--tserver_master_addrs=127.11.175.254:45535
--webserver_port=38385
--webserver_interface=127.11.175.195
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab
--principal=oryx/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.175.254:45535,127.11.175.253:36139
--tserver_master_addrs=127.11.175.254:45535,127.11.175.253:36139,127.11.175.252:36403 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:57:10.702749 19111 heartbeater.cc:503] Master 127.11.175.254:45535 was elected leader, sending a full tablet report...
W20260619 14:57:10.816999 19268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:10.817291 19268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:10.817444 19268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:10.824971 19268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:10.825095 19268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:10.825275 19268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:57:10.836128 19268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab
--principal=oryx/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:44397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=38385
--tserver_master_addrs=127.11.175.254:45535,127.11.175.253:36139,127.11.175.252:36403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:10.837646 19268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:10.839143 19268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:10.853657 19275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:10.854197 19274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:10.854952 19277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:10.855599 19268 server_base.cc:1034] running on GCE node
I20260619 14:57:10.856259 19268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:10.857496 19268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:10.858770 19268 hybrid_clock.cc:648] HybridClock initialized: now 1781881030858705 us; error 43 us; skew 500 ppm
I20260619 14:57:10.859169 19268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:10 dist-test-slave-76rk krb5kdc[18097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881030, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:10.862884 19268 init.cc:377] Logged in from keytab as oryx/127.11.175.195@KRBTEST.COM (short username oryx)
I20260619 14:57:10.865020 19268 webserver.cc:460] Webserver started at http://127.11.175.195:38385/ using document root <none> and password file <none>
I20260619 14:57:10.866011 19268 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:10.866187 19268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:10.872985 19268 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 14:57:10.875887 19284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:10.877472 19268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 14:57:10.877660 19268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "23a1f3ae85a54085b8628e1b77ffc780"
format_stamp: "Formatted at 2026-06-19 14:57:07 on dist-test-slave-76rk"
server_key: "53acd6c101c6261224539f49501fe0a9"
server_key_iv: "e4bc8b626b2aaed7fb1fca2857ec0906"
server_key_version: "encryptionkey@0"
I20260619 14:57:10.878775 19268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:10.896179 19268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:10.900985 19268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:10.901294 19268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:10.902544 19268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:10.904004 19268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:10.904094 19268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:10.904234 19268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:10.904311 19268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:10.937163 19268 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:44397
I20260619 14:57:10.937278 19396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:44397 every 8 connection(s)
I20260619 14:57:10.938725 19268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:57:10.948021 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 19268
Jun 19 14:57:10 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881030, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:10.967630 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.941724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:44647 (local address 127.11.175.254:45535)
0619 14:57:10.942299 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:10.942309 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:10.943846 (+ 1537us) server_negotiation.cc:407] Connection header received
0619 14:57:10.947189 (+ 3343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:10.947195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:10.947291 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:10.947413 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:10.950188 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.951587 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.953484 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.954440 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.954577 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.958801 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:10.958811 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:10.958819 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:10.958875 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:10.963551 (+ 4676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.964588 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.964597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.964602 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.964789 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.965240 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.965247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.965252 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.965655 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:10.965807 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:10.967025 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0619 14:57:10.967261 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":269,"threads_started":1}
I20260619 14:57:10.969794 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.942401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.195:44647)
0619 14:57:10.943688 (+ 1287us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:10.943724 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:10.946829 (+ 3105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:10.947740 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:10.947757 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:10.948939 (+ 1182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:10.949983 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.950013 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.951778 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.951783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.953329 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.953342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.954708 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.954716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.955061 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.955845 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:10.955873 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:10.958502 (+ 2629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:10.963713 (+ 5211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:10.963726 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:10.963751 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:10.964450 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:10.964894 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:10.964899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:10.964902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:10.965121 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:10.965777 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:10.965785 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:10.966493 (+ 708us) client_negotiation.cc:769] Sending connection context
0619 14:57:10.966939 (+ 446us) client_negotiation.cc:240] Negotiation successful
0619 14:57:10.967493 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"thread_start_us":23857,"threads_started":1}
I20260619 14:57:10.980341 19398 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45535
I20260619 14:57:10.980784 19398 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:10.981886 19398 heartbeater.cc:511] Master 127.11.175.254:45535 requested a full tablet report, sending...
I20260619 14:57:10.983795 18141 ts_manager.cc:194] Re-registered known tserver with Master: 23a1f3ae85a54085b8628e1b77ffc780 (127.11.175.195:44397)
Jun 19 14:57:10 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881030, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.253@KRBTEST.COM
I20260619 14:57:10.985001 18141 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:44647
I20260619 14:57:10.996949 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.944661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:36139 (local address 127.11.175.195:54233)
0619 14:57:10.966086 (+ 21425us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:10.966107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:10.966231 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:10.967876 (+ 1645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:10.967885 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:10.968753 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:10.969239 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.969257 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.971461 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.971469 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.973548 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.973564 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.975587 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.975596 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.977912 (+ 2316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.978820 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:10.978847 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:10.985583 (+ 6736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:10.991845 (+ 6262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:10.991850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:10.991854 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:10.992494 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:10.993137 (+ 643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:10.993144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:10.993150 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:10.993311 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:10.995786 (+ 2475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:10.995792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:10.995976 (+ 184us) client_negotiation.cc:769] Sending connection context
0619 14:57:10.996133 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 14:57:10.996450 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21296,"mutex_wait_us":4,"thread_start_us":20730,"threads_started":1}
I20260619 14:57:10.998144 19406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.944774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:54233 (local address 127.11.175.253:36139)
0619 14:57:10.967261 (+ 22487us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:10.967270 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:10.967296 (+ 26us) server_negotiation.cc:407] Connection header received
0619 14:57:10.967373 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:10.967381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:10.967482 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:10.967617 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:10.969748 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.971278 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.974157 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.975139 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.975307 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.985836 (+ 10529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:10.985845 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:10.985850 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:10.985900 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:10.991689 (+ 5789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.992756 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.992766 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.992772 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.993004 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.995052 (+ 2048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.995058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.995061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.995487 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:10.995700 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:10.997439 (+ 1739us) server_negotiation.cc:299] Negotiation successful
0619 14:57:10.997705 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22328,"thread_start_us":22202,"threads_started":1}
I20260619 14:57:11.000346 19399 heartbeater.cc:347] Connected to a master server at 127.11.175.253:36139
I20260619 14:57:11.000506 19399 heartbeater.cc:464] Registering TS with master...
Jun 19 14:57:10 dist-test-slave-76rk krb5kdc[18097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881030, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.252@KRBTEST.COM
I20260619 14:57:11.001001 19399 heartbeater.cc:511] Master 127.11.175.253:36139 requested a full tablet report, sending...
I20260619 14:57:11.002388 18743 ts_manager.cc:194] Registered new tserver with Master: 23a1f3ae85a54085b8628e1b77ffc780 (127.11.175.195:44397)
I20260619 14:57:11.002876 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.966033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48324 (local address 127.11.175.254:45535)
0619 14:57:10.967861 (+ 1828us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:10.967866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:10.974773 (+ 6907us) server_negotiation.cc:407] Connection header received
0619 14:57:10.975086 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:10.975095 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:10.975191 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:10.975298 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:10.982439 (+ 7141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.984076 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.986791 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.987737 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.987862 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.993004 (+ 5142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:10.993014 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:10.993022 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:10.993100 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:10.997177 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.998213 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.998221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.998226 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.998382 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:10.998856 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:10.998862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:10.998865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:10.999233 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:10.999483 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.002144 (+ 2661us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.002468 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1737,"thread_start_us":12689,"threads_started":1}
I20260619 14:57:11.012280 19402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.941829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:40721 (local address 127.11.175.252:36403)
0619 14:57:10.942992 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:10.942997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:10.969518 (+ 26521us) server_negotiation.cc:407] Connection header received
0619 14:57:10.969689 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:10.969694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:10.969783 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:10.969897 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:10.973128 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.974031 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.989283 (+ 15252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.990287 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.990462 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.002296 (+ 11834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:11.002304 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:11.002310 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:11.002357 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:11.009137 (+ 6780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.009899 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.009904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.009906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.010102 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.010578 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.010582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.010584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.011002 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:11.011208 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.011629 (+ 421us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.011884 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1071,"thread_start_us":978,"threads_started":1}
I20260619 14:57:11.012315 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.942990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:36403 (local address 127.11.175.195:40721)
0619 14:57:10.969417 (+ 26427us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:10.969440 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:10.969566 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:10.971573 (+ 2007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:10.971581 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:10.972453 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:10.972929 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.972947 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.981141 (+ 8194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.981152 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.982796 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:10.982812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.991937 (+ 9125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:10.991943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:10.992169 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:10.993237 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:10.993256 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:11.002064 (+ 8808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:11.009338 (+ 7274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:11.009344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:11.009349 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:11.009750 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:11.010235 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:11.010241 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:11.010245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:11.010450 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:11.011129 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:11.011151 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:11.011330 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 14:57:11.011472 (+ 142us) client_negotiation.cc:240] Negotiation successful
0619 14:57:11.011846 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22590,"thread_start_us":327,"threads_started":1}
I20260619 14:57:11.013692 19397 heartbeater.cc:347] Connected to a master server at 127.11.175.252:36403
I20260619 14:57:11.013865 19397 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:11.014232 19397 heartbeater.cc:511] Master 127.11.175.252:36403 requested a full tablet report, sending...
I20260619 14:57:11.015486 18931 ts_manager.cc:194] Registered new tserver with Master: 23a1f3ae85a54085b8628e1b77ffc780 (127.11.175.195:44397)
I20260619 14:57:11.015564 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.974406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51390 (local address 127.11.175.253:36139)
0619 14:57:10.995105 (+ 20699us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:10.995111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:10.995127 (+ 16us) server_negotiation.cc:407] Connection header received
0619 14:57:10.995173 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:10.995180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:10.995254 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:10.995478 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:10.997792 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.999342 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.001616 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.002348 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.003796 (+ 1448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.008283 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:11.008290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:11.008296 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:11.008344 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:11.012058 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.013002 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.013008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.013012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.013178 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.013869 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.013873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.013875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.014191 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:11.014450 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.014913 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.015145 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20585,"thread_start_us":20447,"threads_started":1}
I20260619 14:57:11.008689 18141 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:57:11.019080 18141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:57:11.026782 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:10.976167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46840 (local address 127.11.175.252:36403)
0619 14:57:10.994966 (+ 18799us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:10.994973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:10.994994 (+ 21us) server_negotiation.cc:407] Connection header received
0619 14:57:10.995053 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:10.995058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:10.995144 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:10.997748 (+ 2604us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:10.997990 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:10.999321 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.002954 (+ 3633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.004030 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.004863 (+ 833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.019005 (+ 14142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:11.019011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:11.019015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:11.019049 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:11.022200 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.023956 (+ 1756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.023962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.023965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.024163 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.024793 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.024797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.024801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.025219 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:11.025455 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.025977 (+ 522us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.026261 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18660,"thread_start_us":18510,"threads_started":1}
I20260619 14:57:11.077109 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.044456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.11.175.193:39773)
0619 14:57:11.068122 (+ 23666us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:11.068135 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:11.068174 (+ 39us) server_negotiation.cc:407] Connection header received
0619 14:57:11.068302 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:11.068313 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:11.068667 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:11.068939 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:11.069998 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.071750 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.074046 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.075457 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.075652 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.075857 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.076479 (+ 622us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.076695 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23433,"thread_start_us":23324,"threads_started":1}
I20260619 14:57:11.077435 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.044149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39773 (local address 127.0.0.1:45410)
0619 14:57:11.066998 (+ 22849us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:11.067027 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:11.067236 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:11.068974 (+ 1738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:11.068985 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:11.069042 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:11.069671 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:11.069699 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.071947 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.071952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:11.073850 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:11.073869 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.076126 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.076129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:11.076278 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.076303 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 14:57:11.076429 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 14:57:11.076718 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22629,"mutex_wait_us":112,"thread_start_us":25636,"threads_started":1}
I20260619 14:57:11.080603 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.044870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:39947 (local address 127.0.0.1:51024)
0619 14:57:11.070949 (+ 26079us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:11.070967 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:11.071177 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:11.072066 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:11.072069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:11.072094 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:11.072485 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:11.072492 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.074112 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.074118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:11.076663 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:11.076679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.079766 (+ 3087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.079773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:11.080035 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.080072 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 14:57:11.080174 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 14:57:11.080323 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25970,"thread_start_us":22508,"threads_started":1}
I20260619 14:57:11.081853 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.044353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:44397 (local address 127.0.0.1:54182)
0619 14:57:11.070152 (+ 25799us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:11.070184 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:11.070376 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:11.072697 (+ 2321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:11.072702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:11.072729 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:11.073148 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:11.073161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.075929 (+ 2768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.075934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:11.078816 (+ 2882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:11.078832 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.080792 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.080801 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:11.081167 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.081209 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 14:57:11.081331 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 14:57:11.081531 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25625,"thread_start_us":26342,"threads_started":1}
I20260619 14:57:11.081820 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.045134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.11.175.194:39947)
0619 14:57:11.069548 (+ 24414us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:11.069561 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:11.071071 (+ 1510us) server_negotiation.cc:407] Connection header received
0619 14:57:11.071384 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:11.071392 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:11.071632 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:11.071791 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:11.072658 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.073988 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.076926 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.079169 (+ 2243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.079393 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.079575 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.081212 (+ 1637us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.081520 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24246,"thread_start_us":24141,"threads_started":1}
I20260619 14:57:11.082238 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.045263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.11.175.195:44397)
0619 14:57:11.066019 (+ 20756us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:11.066033 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:11.070340 (+ 4307us) server_negotiation.cc:407] Connection header received
0619 14:57:11.070570 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:11.070580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:11.070917 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:11.071142 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:11.073366 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.075631 (+ 2265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.079046 (+ 3415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.080552 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.080974 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.081169 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.081453 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.081637 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20504,"thread_start_us":20801,"threads_started":1}
I20260619 14:57:11.083868 19045 tablet_service.cc:1467] Processing CreateTablet for tablet d1d78f30cc1e4216ba2dff538bc52a9a (DEFAULT_TABLE table=test-table [id=26fe37fe1868464a95400ce04dac965b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:11.085484 19045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1d78f30cc1e4216ba2dff538bc52a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:11.085795 19190 tablet_service.cc:1467] Processing CreateTablet for tablet d1d78f30cc1e4216ba2dff538bc52a9a (DEFAULT_TABLE table=test-table [id=26fe37fe1868464a95400ce04dac965b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:11.087296 19190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1d78f30cc1e4216ba2dff538bc52a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:11.090317 19332 tablet_service.cc:1467] Processing CreateTablet for tablet d1d78f30cc1e4216ba2dff538bc52a9a (DEFAULT_TABLE table=test-table [id=26fe37fe1868464a95400ce04dac965b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:11.091719 19332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1d78f30cc1e4216ba2dff538bc52a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:11.106354 19435 tablet_bootstrap.cc:492] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: Bootstrap starting.
I20260619 14:57:11.107004 19434 tablet_bootstrap.cc:492] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: Bootstrap starting.
I20260619 14:57:11.107594 19436 tablet_bootstrap.cc:492] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: Bootstrap starting.
I20260619 14:57:11.110196 19435 tablet_bootstrap.cc:654] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:11.110723 19436 tablet_bootstrap.cc:654] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:11.111586 19435 log.cc:824] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:11.111780 19436 log.cc:824] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:11.113816 19434 tablet_bootstrap.cc:654] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:11.115129 19434 log.cc:824] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:11.115864 19435 tablet_bootstrap.cc:492] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: No bootstrap required, opened a new log
I20260619 14:57:11.116163 19435 ts_tablet_manager.cc:1397] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260619 14:57:11.117318 19436 tablet_bootstrap.cc:492] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: No bootstrap required, opened a new log
I20260619 14:57:11.117594 19436 ts_tablet_manager.cc:1397] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260619 14:57:11.118000 19434 tablet_bootstrap.cc:492] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: No bootstrap required, opened a new log
I20260619 14:57:11.118279 19434 ts_tablet_manager.cc:1397] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.003s
I20260619 14:57:11.126528 19434 raft_consensus.cc:348] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:11.127013 19434 raft_consensus.cc:374] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:11.127136 19434 raft_consensus.cc:729] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca402f97c0461f950e5f721b66d574, State: Initialized, Role: FOLLOWER
I20260619 14:57:11.127465 19435 raft_consensus.cc:348] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:11.127911 19435 raft_consensus.cc:374] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:11.128026 19435 raft_consensus.cc:729] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d8f273611624092b7905bfbb3d2855b, State: Initialized, Role: FOLLOWER
I20260619 14:57:11.127987 19434 consensus_queue.cc:260] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:11.128736 19435 consensus_queue.cc:260] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:11.129025 19436 raft_consensus.cc:348] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:11.129529 19436 raft_consensus.cc:374] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:11.129642 19436 raft_consensus.cc:729] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23a1f3ae85a54085b8628e1b77ffc780, State: Initialized, Role: FOLLOWER
I20260619 14:57:11.130131 19434 ts_tablet_manager.cc:1428] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260619 14:57:11.130471 19256 heartbeater.cc:503] Master 127.11.175.254:45535 was elected leader, sending a full tablet report...
I20260619 14:57:11.131568 19436 consensus_queue.cc:260] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:11.139281 19398 heartbeater.cc:503] Master 127.11.175.254:45535 was elected leader, sending a full tablet report...
I20260619 14:57:11.140169 19436 ts_tablet_manager.cc:1428] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: Time spent starting tablet: real 0.022s user 0.022s sys 0.000s
I20260619 14:57:11.140157 19435 ts_tablet_manager.cc:1428] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: Time spent starting tablet: real 0.024s user 0.009s sys 0.011s
W20260619 14:57:11.164362 19113 tablet.cc:2369] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 14:57:11.191951 19400 tablet.cc:2369] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:57:11.244690 19442 raft_consensus.cc:482] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:57:11.245034 19442 raft_consensus.cc:504] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:11.246978 19442 leader_election.cc:288] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d8f273611624092b7905bfbb3d2855b (127.11.175.193:39773), c4ca402f97c0461f950e5f721b66d574 (127.11.175.194:39947)
I20260619 14:57:11.253914 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.247644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:47419 (local address 127.11.175.194:39947)
0619 14:57:11.247870 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:11.247876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:11.247900 (+ 24us) server_negotiation.cc:407] Connection header received
0619 14:57:11.248037 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:11.248043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:11.248115 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:11.248205 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:11.248819 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.249753 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.251538 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.252845 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.253039 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.253147 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.253448 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.253603 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 14:57:11.253993 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.247253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:39947 (local address 127.11.175.195:47419)
0619 14:57:11.247576 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:11.247596 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:11.247730 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:11.248247 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:11.248254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:11.248285 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:11.248635 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:11.248648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.249886 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.249889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:11.251324 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:11.251337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.253020 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.253024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:11.253257 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.253304 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 14:57:11.253410 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 14:57:11.253598 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":53,"spinlock_wait_cycles":896}
I20260619 14:57:11.254072 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.247369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39773 (local address 127.11.175.195:45109)
0619 14:57:11.247653 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:11.247671 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:11.247735 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:11.248338 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:11.248350 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:11.248408 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:11.248703 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:11.248727 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.249775 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.249783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:11.251391 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:11.251441 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.253151 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.253156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:11.253392 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.253426 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 14:57:11.253504 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 14:57:11.253638 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":124,"spinlock_wait_cycles":2304}
I20260619 14:57:11.254515 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.247456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:45109 (local address 127.11.175.193:39773)
0619 14:57:11.248040 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:11.248044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:11.248060 (+ 16us) server_negotiation.cc:407] Connection header received
0619 14:57:11.248116 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:11.248119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:11.248194 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:11.248289 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:11.248878 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.249637 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.251656 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.253005 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.253165 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.253244 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.253547 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.253678 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395}
I20260619 14:57:11.255332 19210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d78f30cc1e4216ba2dff538bc52a9a" candidate_uuid: "23a1f3ae85a54085b8628e1b77ffc780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca402f97c0461f950e5f721b66d574" is_pre_election: true
I20260619 14:57:11.255610 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d78f30cc1e4216ba2dff538bc52a9a" candidate_uuid: "23a1f3ae85a54085b8628e1b77ffc780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d8f273611624092b7905bfbb3d2855b" is_pre_election: true
I20260619 14:57:11.256021 19210 raft_consensus.cc:2454] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23a1f3ae85a54085b8628e1b77ffc780 in term 0.
I20260619 14:57:11.256140 19065 raft_consensus.cc:2454] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23a1f3ae85a54085b8628e1b77ffc780 in term 0.
I20260619 14:57:11.256918 19288 leader_election.cc:302] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 23a1f3ae85a54085b8628e1b77ffc780, 8d8f273611624092b7905bfbb3d2855b, c4ca402f97c0461f950e5f721b66d574; no voters:
I20260619 14:57:11.257412 19442 raft_consensus.cc:2781] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:57:11.257609 19442 raft_consensus.cc:482] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:57:11.257732 19442 raft_consensus.cc:3037] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:11.259999 19442 raft_consensus.cc:504] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:11.260587 19442 leader_election.cc:288] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [CANDIDATE]: Term 1 election: Requested vote from peers 8d8f273611624092b7905bfbb3d2855b (127.11.175.193:39773), c4ca402f97c0461f950e5f721b66d574 (127.11.175.194:39947)
I20260619 14:57:11.261183 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d78f30cc1e4216ba2dff538bc52a9a" candidate_uuid: "23a1f3ae85a54085b8628e1b77ffc780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d8f273611624092b7905bfbb3d2855b"
I20260619 14:57:11.261185 19210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d78f30cc1e4216ba2dff538bc52a9a" candidate_uuid: "23a1f3ae85a54085b8628e1b77ffc780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca402f97c0461f950e5f721b66d574"
I20260619 14:57:11.261415 19210 raft_consensus.cc:3037] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:11.261438 19065 raft_consensus.cc:3037] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:11.264750 19065 raft_consensus.cc:2454] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23a1f3ae85a54085b8628e1b77ffc780 in term 1.
I20260619 14:57:11.264750 19210 raft_consensus.cc:2454] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23a1f3ae85a54085b8628e1b77ffc780 in term 1.
I20260619 14:57:11.265208 19288 leader_election.cc:302] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23a1f3ae85a54085b8628e1b77ffc780, c4ca402f97c0461f950e5f721b66d574; no voters:
I20260619 14:57:11.265539 19442 raft_consensus.cc:2781] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:11.265980 19442 raft_consensus.cc:686] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 1 LEADER]: Becoming Leader. State: Replica: 23a1f3ae85a54085b8628e1b77ffc780, State: Running, Role: LEADER
I20260619 14:57:11.266534 19442 consensus_queue.cc:237] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:11.272548 18142 catalog_manager.cc:5495] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23a1f3ae85a54085b8628e1b77ffc780 (127.11.175.195). New cstate: current_term: 1 leader_uuid: "23a1f3ae85a54085b8628e1b77ffc780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } health_report { overall_health: UNKNOWN } } }
W20260619 14:57:11.301544 19258 tablet.cc:2369] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:57:11.324204 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.318781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.11.175.195:44397)
0619 14:57:11.318986 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:11.318993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:11.319043 (+ 50us) server_negotiation.cc:407] Connection header received
0619 14:57:11.319184 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:11.319188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:11.319266 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:11.319416 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:57:11.319782 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.320715 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.322056 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.322459 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.322596 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.323101 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:57:11.323594 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:57:11.323744 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.323823 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.323902 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 14:57:11.332707 19210 raft_consensus.cc:1261] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574 [term 1 FOLLOWER]: Refusing update from remote peer 23a1f3ae85a54085b8628e1b77ffc780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:11.332707 19065 raft_consensus.cc:1261] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b [term 1 FOLLOWER]: Refusing update from remote peer 23a1f3ae85a54085b8628e1b77ffc780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:11.333803 19443 consensus_queue.cc:1035] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:11.334203 19442 consensus_queue.cc:1035] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:11.347720 19446 mvcc.cc:205] Tried to move back new op lower bound from 7298584704327192576 to 7298584704071581696. Current Snapshot: MvccSnapshot[applied={T|T < 7298584704327192576}]
I20260619 14:57:11.355858 19448 mvcc.cc:205] Tried to move back new op lower bound from 7298584704327192576 to 7298584704071581696. Current Snapshot: MvccSnapshot[applied={T|T < 7298584704327192576}]
I20260619 14:57:11.365387 18142 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:11.365656 18142 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:11.372344 18142 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e70967bec8d64eb780c54c87c293f1e4: Sending DeleteTablet for 3 replicas of tablet d1d78f30cc1e4216ba2dff538bc52a9a
I20260619 14:57:11.373546 19045 tablet_service.cc:1514] Processing DeleteTablet for tablet d1d78f30cc1e4216ba2dff538bc52a9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:11 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:45410
I20260619 14:57:11.373939 19332 tablet_service.cc:1514] Processing DeleteTablet for tablet d1d78f30cc1e4216ba2dff538bc52a9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:11 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:54182
I20260619 14:57:11.374826 19455 tablet_replica.cc:330] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: stopping tablet replica
I20260619 14:57:11.375293 19455 raft_consensus.cc:2229] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:11.375855 19455 raft_consensus.cc:2258] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:11.376345 19190 tablet_service.cc:1514] Processing DeleteTablet for tablet d1d78f30cc1e4216ba2dff538bc52a9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:11 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:51024
I20260619 14:57:11.378832 19455 ts_tablet_manager.cc:1905] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:11.382625 19456 tablet_replica.cc:330] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: stopping tablet replica
I20260619 14:57:11.383654 19456 raft_consensus.cc:2229] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:57:11.384979 19456 raft_consensus.cc:2258] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:11.387490 19458 tablet_replica.cc:330] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: stopping tablet replica
I20260619 14:57:11.388180 19458 raft_consensus.cc:2229] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:11.388784 19455 ts_tablet_manager.cc:1918] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:11.388971 19455 log.cc:1196] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/d1d78f30cc1e4216ba2dff538bc52a9a
I20260619 14:57:11.390527 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 18110
I20260619 14:57:11.391258 19455 ts_tablet_manager.cc:1939] T d1d78f30cc1e4216ba2dff538bc52a9a P 8d8f273611624092b7905bfbb3d2855b: Deleting consensus metadata
I20260619 14:57:11.391462 19458 raft_consensus.cc:2258] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:11.392046 18172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:11.394068 19458 ts_tablet_manager.cc:1905] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:11.398911 19456 ts_tablet_manager.cc:1905] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:11.401952 19458 ts_tablet_manager.cc:1918] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:11.402285 19458 log.cc:1196] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/d1d78f30cc1e4216ba2dff538bc52a9a
I20260619 14:57:11.403759 19458 ts_tablet_manager.cc:1939] T d1d78f30cc1e4216ba2dff538bc52a9a P c4ca402f97c0461f950e5f721b66d574: Deleting consensus metadata
I20260619 14:57:11.411958 19456 ts_tablet_manager.cc:1918] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:11.412120 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.377046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.11.175.253:36139)
0619 14:57:11.377195 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:11.377201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:11.389716 (+ 12515us) server_negotiation.cc:407] Connection header received
0619 14:57:11.391916 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:11.391926 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:11.392056 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:11.392210 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:11.395219 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.396743 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.399169 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.399866 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.399977 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.404861 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:11.404868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:11.404876 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:11.404931 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:11.408690 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.409646 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.409652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.409656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.409808 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.410275 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.410280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.410283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.410687 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:11.410927 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.411486 (+ 559us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.411765 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 14:57:11.412170 19456 log.cc:1196] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/d1d78f30cc1e4216ba2dff538bc52a9a
I20260619 14:57:11.412786 19456 ts_tablet_manager.cc:1939] T d1d78f30cc1e4216ba2dff538bc52a9a P 23a1f3ae85a54085b8628e1b77ffc780: Deleting consensus metadata
I20260619 14:57:11.429934 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.389148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46846 (local address 127.11.175.252:36403)
0619 14:57:11.389416 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:11.389424 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:11.414812 (+ 25388us) server_negotiation.cc:407] Connection header received
0619 14:57:11.414986 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:11.414992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:11.415074 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:11.415183 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:11.416571 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.417279 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.418603 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.419243 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.419345 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.423130 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:11.423138 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:11.423143 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:11.423189 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:11.426825 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.427633 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.427639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.427643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.427781 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.428230 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.428236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.428240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.428578 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:11.428805 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.429351 (+ 546us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.429602 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 14:57:11.547798 18126 catalog_manager.cc:4841] TS 8d8f273611624092b7905bfbb3d2855b (127.11.175.193:39773): tablet d1d78f30cc1e4216ba2dff538bc52a9a (table test-table [id=26fe37fe1868464a95400ce04dac965b]) successfully deleted
I20260619 14:57:11.548064 18128 catalog_manager.cc:4841] TS 23a1f3ae85a54085b8628e1b77ffc780 (127.11.175.195:44397): tablet d1d78f30cc1e4216ba2dff538bc52a9a (table test-table [id=26fe37fe1868464a95400ce04dac965b]) successfully deleted
I20260619 14:57:11.548298 18129 catalog_manager.cc:4841] TS c4ca402f97c0461f950e5f721b66d574 (127.11.175.194:39947): tablet d1d78f30cc1e4216ba2dff538bc52a9a (table test-table [id=26fe37fe1868464a95400ce04dac965b]) successfully deleted
I20260619 14:57:11.548569 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 18110
I20260619 14:57:11.595598 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.576681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40266 (local address 127.11.175.253:36139)
0619 14:57:11.576969 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:11.576980 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:11.578120 (+ 1140us) server_negotiation.cc:407] Connection header received
0619 14:57:11.578334 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:11.578341 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:11.578422 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:11.578538 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:11.580752 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.581667 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.583758 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.584442 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.584583 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.588722 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:11.588728 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:11.588732 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:11.588770 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:11.592353 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.593493 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.593499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.593502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.593643 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.594023 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.594028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.594031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.594371 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:11.594621 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.595009 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.595226 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 14:57:11.597599 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:11.578217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.11.175.252:36403)
0619 14:57:11.578405 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:11.578411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:11.578987 (+ 576us) server_negotiation.cc:407] Connection header received
0619 14:57:11.579132 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:11.579139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:11.579232 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:11.579368 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:11.581525 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.582460 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.584277 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:11.584951 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:11.585083 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:11.589586 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:11.589592 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:11.589595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:11.589633 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:11.592956 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.594808 (+ 1852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.594813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.594816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.595045 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:11.595507 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:11.595512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:11.595515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:11.596177 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:11.596413 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:11.597196 (+ 783us) server_negotiation.cc:299] Negotiation successful
0619 14:57:11.597365 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 14:57:12.356778 19467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.355847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.193:51933)
0619 14:57:12.356389 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.356557 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":288,"threads_started":1}
I20260619 14:57:12.356812 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.355836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.195:48381)
0619 14:57:12.356389 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.356566 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":294,"threads_started":1}
W20260619 14:57:12.357230 19398 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:45535 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
W20260619 14:57:12.357370 19111 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:45535 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
I20260619 14:57:12.361779 19468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.359837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.194:36853)
0619 14:57:12.361407 (+ 1570us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.361544 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1405,"thread_start_us":1282,"threads_started":1}
W20260619 14:57:12.362196 19256 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:45535 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
I20260619 14:57:12.872648 19469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e70967bec8d64eb780c54c87c293f1e4)
I20260619 14:57:12.873065 19469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } }
I20260619 14:57:12.874761 19469 leader_election.cc:288] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e70967bec8d64eb780c54c87c293f1e4 (127.11.175.254:45535), 70fba17797984460af4e7336695f1cec (127.11.175.253:36139)
I20260619 14:57:12.879729 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.874810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:40364)
0619 14:57:12.879252 (+ 4442us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.879482 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4260,"thread_start_us":4456,"threads_started":1}
W20260619 14:57:12.880131 18917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
I20260619 14:57:12.880297 19470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e70967bec8d64eb780c54c87c293f1e4)
I20260619 14:57:12.880613 19470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } }
I20260619 14:57:12.882052 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.881516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:40376)
0619 14:57:12.881719 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.881833 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 14:57:12.882295 19470 leader_election.cc:288] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e70967bec8d64eb780c54c87c293f1e4 (127.11.175.254:45535), 706f093f7bd749bfa4a548ab8a7d8953 (127.11.175.252:36403)
W20260619 14:57:12.882298 18917 leader_election.cc:334] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e70967bec8d64eb780c54c87c293f1e4 (127.11.175.254:45535): Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
I20260619 14:57:12.887867 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.883839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:40382)
0619 14:57:12.887310 (+ 3471us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.887531 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3353,"thread_start_us":3503,"threads_started":1}
W20260619 14:57:12.888218 18729 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
I20260619 14:57:12.890352 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.875011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:36139 (local address 127.0.0.1:40280)
0619 14:57:12.879635 (+ 4624us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.879660 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:12.879858 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:12.883396 (+ 3538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:12.883401 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:12.883432 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:12.883759 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:12.883776 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.885808 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.885817 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:12.888265 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:12.888279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.889431 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.889438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:12.889680 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:12.889736 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 14:57:12.889847 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 14:57:12.890001 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4465,"thread_start_us":3839,"threads_started":1}
I20260619 14:57:12.891001 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.880221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40280 (local address 127.11.175.253:36139)
0619 14:57:12.883062 (+ 2841us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:12.883071 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:12.883093 (+ 22us) server_negotiation.cc:407] Connection header received
0619 14:57:12.883165 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:12.883171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:12.883249 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:12.883370 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:12.883954 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.885623 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.888406 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.889264 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.889430 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:12.889508 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:12.890375 (+ 867us) server_negotiation.cc:299] Negotiation successful
0619 14:57:12.890561 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2612,"thread_start_us":3316,"threads_started":1}
I20260619 14:57:12.891137 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.890689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:40396)
0619 14:57:12.890864 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.890956 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 14:57:12.891382 18729 leader_election.cc:334] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e70967bec8d64eb780c54c87c293f1e4 (127.11.175.254:45535): Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
I20260619 14:57:12.892009 18753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "706f093f7bd749bfa4a548ab8a7d8953" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "70fba17797984460af4e7336695f1cec" is_pre_election: true
I20260619 14:57:12.892385 18753 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 706f093f7bd749bfa4a548ab8a7d8953 in term 1.
I20260619 14:57:12.893013 18918 leader_election.cc:302] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 706f093f7bd749bfa4a548ab8a7d8953, 70fba17797984460af4e7336695f1cec; no voters: e70967bec8d64eb780c54c87c293f1e4
I20260619 14:57:12.893440 19469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 14:57:12.893579 19469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e70967bec8d64eb780c54c87c293f1e4)
I20260619 14:57:12.893675 19469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 1 FOLLOWER]: Advancing to term 2
I20260619 14:57:12.894264 19475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.882544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:36403 (local address 127.0.0.1:60702)
0619 14:57:12.887094 (+ 4550us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.887117 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:12.887262 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:12.887648 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:12.887653 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:12.887687 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:12.888096 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:12.888111 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.889954 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.889958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:12.892125 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:12.892136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.893495 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.893501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:12.893674 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:12.893698 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 14:57:12.893796 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 14:57:12.893927 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4418,"thread_start_us":4275,"threads_started":1}
I20260619 14:57:12.894331 19476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.882710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.11.175.252:36403)
0619 14:57:12.883376 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:12.883384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:12.887219 (+ 3835us) server_negotiation.cc:407] Connection header received
0619 14:57:12.887410 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:12.887415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:12.887499 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:12.887612 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:12.888306 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.889787 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.892270 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.893306 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.893506 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:12.893585 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:12.893819 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 14:57:12.893953 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":383,"threads_started":1}
I20260619 14:57:12.895056 18941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70fba17797984460af4e7336695f1cec" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "706f093f7bd749bfa4a548ab8a7d8953" is_pre_election: true
I20260619 14:57:12.896184 19469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } }
I20260619 14:57:12.896636 18941 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70fba17797984460af4e7336695f1cec in current term 2: Already voted for candidate 706f093f7bd749bfa4a548ab8a7d8953 in this term.
I20260619 14:57:12.896839 19469 leader_election.cc:288] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [CANDIDATE]: Term 2 election: Requested vote from peers e70967bec8d64eb780c54c87c293f1e4 (127.11.175.254:45535), 70fba17797984460af4e7336695f1cec (127.11.175.253:36139)
I20260619 14:57:12.897346 18753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "706f093f7bd749bfa4a548ab8a7d8953" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "70fba17797984460af4e7336695f1cec"
I20260619 14:57:12.897537 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.896959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:40406)
0619 14:57:12.897318 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.897403 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247}
I20260619 14:57:12.897593 18753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 1 FOLLOWER]: Advancing to term 2
I20260619 14:57:12.897545 18730 leader_election.cc:302] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70fba17797984460af4e7336695f1cec; no voters: 706f093f7bd749bfa4a548ab8a7d8953, e70967bec8d64eb780c54c87c293f1e4
I20260619 14:57:12.898582 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.898244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:40416)
0619 14:57:12.898384 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.898471 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 14:57:12.898751 18917 leader_election.cc:334] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e70967bec8d64eb780c54c87c293f1e4 (127.11.175.254:45535): Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
I20260619 14:57:12.899853 18753 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 706f093f7bd749bfa4a548ab8a7d8953 in term 2.
I20260619 14:57:12.900060 19470 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 14:57:12.900245 18918 leader_election.cc:302] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 706f093f7bd749bfa4a548ab8a7d8953, 70fba17797984460af4e7336695f1cec; no voters: e70967bec8d64eb780c54c87c293f1e4
I20260619 14:57:12.900733 19469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 14:57:12.901216 19469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [term 2 LEADER]: Becoming Leader. State: Replica: 706f093f7bd749bfa4a548ab8a7d8953, State: Running, Role: LEADER
I20260619 14:57:12.901630 19469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } }
I20260619 14:57:12.903692 19479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 706f093f7bd749bfa4a548ab8a7d8953. Latest consensus state: current_term: 2 leader_uuid: "706f093f7bd749bfa4a548ab8a7d8953" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:12.904135 19479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:12.904789 19481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:57:12.908468 19481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:57:12.909325 19481 catalog_manager.cc:1259] Loaded cluster ID: 0e8d9b9cd3b241228c5ff51005f028bc
I20260619 14:57:12.909425 19481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:57:12.910604 19481 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:57:12.911486 19481 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Loaded TSK: 0
I20260619 14:57:12.912340 19481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 14:57:12.931834 18931 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:57:12.938967 18931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:57:12.943109 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.942641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:40428)
0619 14:57:12.942819 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.942919 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 14:57:12.943181 18753 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [term 2 FOLLOWER]: Refusing update from remote peer 706f093f7bd749bfa4a548ab8a7d8953: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260619 14:57:12.944062 19479 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 14:57:12.944204 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.943763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:40444)
0619 14:57:12.943957 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.944039 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 14:57:12.944406 18917 consensus_peers.cc:487] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 -> Peer e70967bec8d64eb780c54c87c293f1e4 (127.11.175.254:45535): Couldn't send request to peer e70967bec8d64eb780c54c87c293f1e4. Status: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 14:57:12.946853 19470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 706f093f7bd749bfa4a548ab8a7d8953. Latest consensus state: current_term: 2 leader_uuid: "706f093f7bd749bfa4a548ab8a7d8953" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:12.947157 19470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:57:12.948809 19479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "706f093f7bd749bfa4a548ab8a7d8953" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:12.949105 19479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:12.949359 19470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "706f093f7bd749bfa4a548ab8a7d8953" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e70967bec8d64eb780c54c87c293f1e4" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45535 } } peers { permanent_uuid: "70fba17797984460af4e7336695f1cec" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 36139 } attrs { promote: false } } peers { permanent_uuid: "706f093f7bd749bfa4a548ab8a7d8953" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 36403 } attrs { promote: false } } }
I20260619 14:57:12.949666 19470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70fba17797984460af4e7336695f1cec [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:57:12.966459 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.957132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:44397 (local address 127.0.0.1:55240)
0619 14:57:12.957274 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.957293 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:12.957382 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:12.958148 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:12.958153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:12.958177 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:12.958440 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:12.958456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.960565 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.960574 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:12.963233 (+ 2659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:12.963254 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.965379 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.965388 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:12.965683 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:12.965744 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 14:57:12.965826 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 14:57:12.966023 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 14:57:12.968458 19488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.957274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55240 (local address 127.11.175.195:44397)
0619 14:57:12.957810 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:12.957818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:12.957835 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:57:12.957915 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:12.957919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:12.957999 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:12.958630 (+ 631us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:12.958871 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.960340 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.963502 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.965164 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.967621 (+ 2457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:12.967751 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:12.967814 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 14:57:12.968010 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":301,"threads_started":1}
I20260619 14:57:12.969633 19332 tablet_service.cc:1467] Processing CreateTablet for tablet de66438232c24d319e2b53f09d55be12 (DEFAULT_TABLE table=test-table [id=fc500109c2bc47568ba45bcf306b4b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:12.970242 19332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de66438232c24d319e2b53f09d55be12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:12.976888 19492 tablet_bootstrap.cc:492] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780: Bootstrap starting.
I20260619 14:57:12.978559 19492 tablet_bootstrap.cc:654] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:12.984143 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.958324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45676 (local address 127.11.175.194:39947)
0619 14:57:12.976040 (+ 17716us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:12.976050 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:12.976079 (+ 29us) server_negotiation.cc:407] Connection header received
0619 14:57:12.976176 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:12.976184 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:12.976291 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:12.977158 (+ 867us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:12.977360 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.978865 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.981751 (+ 2886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.983033 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.983215 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:12.983295 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:12.983699 (+ 404us) server_negotiation.cc:299] Negotiation successful
0619 14:57:12.983844 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17527,"thread_start_us":18076,"threads_started":1}
I20260619 14:57:12.984172 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.958164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:39947 (local address 127.0.0.1:45676)
0619 14:57:12.967182 (+ 9018us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.967206 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:12.967330 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:12.976601 (+ 9271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:12.976610 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:12.976649 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:12.976963 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:12.976978 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.979040 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.979046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:12.981570 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:12.981593 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.983215 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.983222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:12.983504 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:12.983549 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 14:57:12.983672 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 14:57:12.983824 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8448,"thread_start_us":14988,"threads_started":1}
I20260619 14:57:12.984220 19492 tablet_bootstrap.cc:492] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780: No bootstrap required, opened a new log
I20260619 14:57:12.984355 19492 ts_tablet_manager.cc:1397] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260619 14:57:12.985194 19190 tablet_service.cc:1467] Processing CreateTablet for tablet de66438232c24d319e2b53f09d55be12 (DEFAULT_TABLE table=test-table [id=fc500109c2bc47568ba45bcf306b4b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:12.985212 19492 raft_consensus.cc:348] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:12.985509 19492 raft_consensus.cc:374] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:12.985651 19190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de66438232c24d319e2b53f09d55be12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:12.985637 19492 raft_consensus.cc:729] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23a1f3ae85a54085b8628e1b77ffc780, State: Initialized, Role: FOLLOWER
I20260619 14:57:12.985888 19492 consensus_queue.cc:260] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:12.986821 19492 ts_tablet_manager.cc:1428] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260619 14:57:12.987542 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.957671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39773 (local address 127.0.0.1:56656)
0619 14:57:12.957812 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.957831 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:12.957928 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:12.981181 (+ 23253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:12.981186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:12.981211 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:12.981490 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:12.981498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.982937 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.982945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:12.984978 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:12.984993 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.986740 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.986747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:12.986956 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:12.987004 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 14:57:12.987087 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 14:57:12.987232 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":4}
I20260619 14:57:12.991394 19397 heartbeater.cc:503] Master 127.11.175.252:36403 was elected leader, sending a full tablet report...
I20260619 14:57:12.994094 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.958091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.11.175.193:39773)
0619 14:57:12.980900 (+ 22809us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:12.980904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:12.980920 (+ 16us) server_negotiation.cc:407] Connection header received
0619 14:57:12.980975 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:12.980978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:12.981044 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:12.981162 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:12.981659 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.982778 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.985214 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:12.986473 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:12.992055 (+ 5582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:12.992159 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:12.993444 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0619 14:57:12.993643 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22695,"thread_start_us":22601,"threads_started":1}
I20260619 14:57:12.994931 19045 tablet_service.cc:1467] Processing CreateTablet for tablet de66438232c24d319e2b53f09d55be12 (DEFAULT_TABLE table=test-table [id=fc500109c2bc47568ba45bcf306b4b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:12.995134 19496 tablet_bootstrap.cc:492] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574: Bootstrap starting.
I20260619 14:57:12.995155 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.987550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.195:58239)
0619 14:57:12.994682 (+ 7132us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.994882 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6986,"thread_start_us":6910,"threads_started":1}
I20260619 14:57:12.995412 19045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de66438232c24d319e2b53f09d55be12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:12.996191 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:12.995793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.195:36781)
0619 14:57:12.995976 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:12.996064 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 14:57:12.996994 19496 tablet_bootstrap.cc:654] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:12.998987 19496 tablet_bootstrap.cc:492] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574: No bootstrap required, opened a new log
I20260619 14:57:12.999132 19496 ts_tablet_manager.cc:1397] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 14:57:13.000092 19496 raft_consensus.cc:348] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:13.000294 19496 raft_consensus.cc:374] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:13.000393 19496 raft_consensus.cc:729] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ca402f97c0461f950e5f721b66d574, State: Initialized, Role: FOLLOWER
I20260619 14:57:13.000707 19496 consensus_queue.cc:260] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:13.001147 19498 tablet_bootstrap.cc:492] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b: Bootstrap starting.
I20260619 14:57:13.003593 19498 tablet_bootstrap.cc:654] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:13.003659 19496 ts_tablet_manager.cc:1428] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260619 14:57:13.004422 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.003423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.194:35731)
0619 14:57:13.004004 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.004176 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"threads_started":1}
I20260619 14:57:13.005441 19255 heartbeater.cc:503] Master 127.11.175.252:36403 was elected leader, sending a full tablet report...
I20260619 14:57:13.006134 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.005728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.194:53351)
0619 14:57:13.005865 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.005975 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 14:57:13.007143 19498 tablet_bootstrap.cc:492] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b: No bootstrap required, opened a new log
I20260619 14:57:13.007360 19498 ts_tablet_manager.cc:1397] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 14:57:13.008044 19498 raft_consensus.cc:348] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:13.008217 19498 raft_consensus.cc:374] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:13.008344 19498 raft_consensus.cc:729] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d8f273611624092b7905bfbb3d2855b, State: Initialized, Role: FOLLOWER
I20260619 14:57:13.008579 19498 consensus_queue.cc:260] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:13.009261 19498 ts_tablet_manager.cc:1428] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 14:57:13.010918 19503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.009586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.193:33593)
0619 14:57:13.010431 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.010605 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":290,"threads_started":1}
I20260619 14:57:13.011729 19110 heartbeater.cc:503] Master 127.11.175.252:36403 was elected leader, sending a full tablet report...
I20260619 14:57:13.012066 19503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.011570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.193:38449)
0619 14:57:13.011877 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.011953 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260619 14:57:13.116981 19494 raft_consensus.cc:482] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:57:13.117336 19494 raft_consensus.cc:504] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:13.118053 19494 leader_election.cc:288] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d8f273611624092b7905bfbb3d2855b (127.11.175.193:39773), c4ca402f97c0461f950e5f721b66d574 (127.11.175.194:39947)
I20260619 14:57:13.118625 19209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de66438232c24d319e2b53f09d55be12" candidate_uuid: "23a1f3ae85a54085b8628e1b77ffc780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca402f97c0461f950e5f721b66d574" is_pre_election: true
I20260619 14:57:13.118700 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de66438232c24d319e2b53f09d55be12" candidate_uuid: "23a1f3ae85a54085b8628e1b77ffc780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d8f273611624092b7905bfbb3d2855b" is_pre_election: true
I20260619 14:57:13.118939 19065 raft_consensus.cc:2454] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23a1f3ae85a54085b8628e1b77ffc780 in term 0.
I20260619 14:57:13.118937 19209 raft_consensus.cc:2454] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23a1f3ae85a54085b8628e1b77ffc780 in term 0.
I20260619 14:57:13.119423 19285 leader_election.cc:302] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23a1f3ae85a54085b8628e1b77ffc780, 8d8f273611624092b7905bfbb3d2855b; no voters:
I20260619 14:57:13.119722 19494 raft_consensus.cc:2781] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:57:13.119817 19494 raft_consensus.cc:482] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:57:13.119880 19494 raft_consensus.cc:3037] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:13.121877 19494 raft_consensus.cc:504] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:13.122376 19494 leader_election.cc:288] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [CANDIDATE]: Term 1 election: Requested vote from peers 8d8f273611624092b7905bfbb3d2855b (127.11.175.193:39773), c4ca402f97c0461f950e5f721b66d574 (127.11.175.194:39947)
I20260619 14:57:13.122951 19209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de66438232c24d319e2b53f09d55be12" candidate_uuid: "23a1f3ae85a54085b8628e1b77ffc780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ca402f97c0461f950e5f721b66d574"
I20260619 14:57:13.122988 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de66438232c24d319e2b53f09d55be12" candidate_uuid: "23a1f3ae85a54085b8628e1b77ffc780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d8f273611624092b7905bfbb3d2855b"
I20260619 14:57:13.123184 19209 raft_consensus.cc:3037] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:13.123201 19065 raft_consensus.cc:3037] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:13.125273 19065 raft_consensus.cc:2454] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23a1f3ae85a54085b8628e1b77ffc780 in term 1.
I20260619 14:57:13.125579 19209 raft_consensus.cc:2454] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23a1f3ae85a54085b8628e1b77ffc780 in term 1.
I20260619 14:57:13.125713 19285 leader_election.cc:302] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23a1f3ae85a54085b8628e1b77ffc780, 8d8f273611624092b7905bfbb3d2855b; no voters:
I20260619 14:57:13.126025 19494 raft_consensus.cc:2781] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:13.126462 19494 raft_consensus.cc:686] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 1 LEADER]: Becoming Leader. State: Replica: 23a1f3ae85a54085b8628e1b77ffc780, State: Running, Role: LEADER
I20260619 14:57:13.126930 19494 consensus_queue.cc:237] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } } peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } }
I20260619 14:57:13.127540 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.127063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.195:48909)
0619 14:57:13.127237 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.127358 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 14:57:13.129745 18930 catalog_manager.cc:5495] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23a1f3ae85a54085b8628e1b77ffc780 (127.11.175.195). New cstate: current_term: 1 leader_uuid: "23a1f3ae85a54085b8628e1b77ffc780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a1f3ae85a54085b8628e1b77ffc780" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 } health_report { overall_health: UNKNOWN } } }
I20260619 14:57:13.164391 19488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.159518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55252 (local address 127.11.175.195:44397)
0619 14:57:13.159703 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:13.159708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:13.159813 (+ 105us) server_negotiation.cc:407] Connection header received
0619 14:57:13.160077 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:13.160081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:13.160151 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:13.160231 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:57:13.160613 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:13.161305 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:13.162774 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:13.163213 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:13.163330 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:13.163673 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:57:13.163806 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:57:13.163889 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:13.164037 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 14:57:13.164105 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 14:57:13.167801 19209 raft_consensus.cc:1261] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574 [term 1 FOLLOWER]: Refusing update from remote peer 23a1f3ae85a54085b8628e1b77ffc780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:13.167944 19065 raft_consensus.cc:1261] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b [term 1 FOLLOWER]: Refusing update from remote peer 23a1f3ae85a54085b8628e1b77ffc780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:13.168412 19494 consensus_queue.cc:1035] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ca402f97c0461f950e5f721b66d574" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:13.168727 19504 consensus_queue.cc:1035] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d8f273611624092b7905bfbb3d2855b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:13.172232 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.171558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.194:59331)
0619 14:57:13.171788 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.171943 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 14:57:13.172844 19503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.172190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.193:51757)
0619 14:57:13.172470 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.172602 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260619 14:57:13.178705 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.178080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.194:44935)
0619 14:57:13.178347 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.178464 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260619 14:57:13.183826 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.183286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.195:39859)
0619 14:57:13.183565 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.183672 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260619 14:57:13.189265 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.188525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.195:39081)
0619 14:57:13.188756 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.188850 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260619 14:57:13.190670 19508 mvcc.cc:205] Tried to move back new op lower bound from 7298584711848587264 to 7298584711690039296. Current Snapshot: MvccSnapshot[applied={T|T < 7298584711848587264}]
I20260619 14:57:13.191428 19503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.190928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.11.175.193:54329)
0619 14:57:13.191149 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.191240 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260619 14:57:13.193679 18930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:13.193934 18930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:13.199679 18930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 706f093f7bd749bfa4a548ab8a7d8953: Sending DeleteTablet for 3 replicas of tablet de66438232c24d319e2b53f09d55be12
I20260619 14:57:13.200665 19332 tablet_service.cc:1514] Processing DeleteTablet for tablet de66438232c24d319e2b53f09d55be12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:13 UTC) from {username='oryx', principal='oryx/127.11.175.252@KRBTEST.COM'} at 127.0.0.1:55240
I20260619 14:57:13.201035 19045 tablet_service.cc:1514] Processing DeleteTablet for tablet de66438232c24d319e2b53f09d55be12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:13 UTC) from {username='oryx', principal='oryx/127.11.175.252@KRBTEST.COM'} at 127.0.0.1:56656
I20260619 14:57:13.201146 19190 tablet_service.cc:1514] Processing DeleteTablet for tablet de66438232c24d319e2b53f09d55be12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:13 UTC) from {username='oryx', principal='oryx/127.11.175.252@KRBTEST.COM'} at 127.0.0.1:45676
I20260619 14:57:13.201386 19515 tablet_replica.cc:330] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780: stopping tablet replica
I20260619 14:57:13.201591 19515 raft_consensus.cc:2229] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:57:13.202056 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 18966
I20260619 14:57:13.202021 19515 raft_consensus.cc:2258] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:13.203186 19515 ts_tablet_manager.cc:1905] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:13.208371 19515 ts_tablet_manager.cc:1918] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:13.208523 19515 log.cc:1196] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/de66438232c24d319e2b53f09d55be12
I20260619 14:57:13.208880 19515 ts_tablet_manager.cc:1939] T de66438232c24d319e2b53f09d55be12 P 23a1f3ae85a54085b8628e1b77ffc780: Deleting consensus metadata
I20260619 14:57:13.209822 18917 catalog_manager.cc:4841] TS 23a1f3ae85a54085b8628e1b77ffc780 (127.11.175.195:44397): tablet de66438232c24d319e2b53f09d55be12 (table test-table [id=fc500109c2bc47568ba45bcf306b4b13]) successfully deleted
I20260619 14:57:13.211050 19517 tablet_replica.cc:330] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574: stopping tablet replica
I20260619 14:57:13.211242 19517 raft_consensus.cc:2229] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:13.211385 19516 tablet_replica.cc:330] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b: stopping tablet replica
I20260619 14:57:13.211580 19517 raft_consensus.cc:2258] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:13.211629 19516 raft_consensus.cc:2229] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:13.211962 19516 raft_consensus.cc:2258] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:13.212659 19517 ts_tablet_manager.cc:1905] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:13.213639 19516 ts_tablet_manager.cc:1905] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:13.219259 19517 ts_tablet_manager.cc:1918] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:13.219417 19517 log.cc:1196] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/de66438232c24d319e2b53f09d55be12
I20260619 14:57:13.219857 19517 ts_tablet_manager.cc:1939] T de66438232c24d319e2b53f09d55be12 P c4ca402f97c0461f950e5f721b66d574: Deleting consensus metadata
I20260619 14:57:13.220518 19516 ts_tablet_manager.cc:1918] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:13.220654 19516 log.cc:1196] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/de66438232c24d319e2b53f09d55be12
I20260619 14:57:13.221102 18918 catalog_manager.cc:4841] TS c4ca402f97c0461f950e5f721b66d574 (127.11.175.194:39947): tablet de66438232c24d319e2b53f09d55be12 (table test-table [id=fc500109c2bc47568ba45bcf306b4b13]) successfully deleted
I20260619 14:57:13.221587 19516 ts_tablet_manager.cc:1939] T de66438232c24d319e2b53f09d55be12 P 8d8f273611624092b7905bfbb3d2855b: Deleting consensus metadata
I20260619 14:57:13.222761 18915 catalog_manager.cc:4841] TS 8d8f273611624092b7905bfbb3d2855b (127.11.175.193:39773): tablet de66438232c24d319e2b53f09d55be12 (table test-table [id=fc500109c2bc47568ba45bcf306b4b13]) successfully deleted
I20260619 14:57:13.226155 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.204161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.11.175.193:39773)
0619 14:57:13.204379 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:13.204385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:13.212613 (+ 8228us) server_negotiation.cc:407] Connection header received
0619 14:57:13.212819 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:13.212825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:13.212925 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:13.213051 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:13.214427 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:13.215223 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:13.216464 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:13.216917 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:13.217055 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:13.219864 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:13.219879 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:13.219894 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:13.219956 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:13.223561 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:13.224359 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:13.224367 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:13.224369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:13.224488 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:13.224941 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:13.224945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:13.224946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:13.225297 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:13.225543 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:13.225756 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 14:57:13.225926 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 14:57:13.226876 19105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:13.321509 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 18966
I20260619 14:57:13.339641 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 19122
I20260619 14:57:13.353403 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.340446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.11.175.194:39947)
0619 14:57:13.340695 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:13.340703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:13.340730 (+ 27us) server_negotiation.cc:407] Connection header received
0619 14:57:13.340945 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:13.340956 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:13.341017 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:13.341139 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:13.342436 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:13.343208 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:13.344411 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:13.344818 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:13.344948 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:13.347741 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:13.347757 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:13.347773 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:13.347807 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:13.350515 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:13.351345 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:13.351353 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:13.351355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:13.351477 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:13.351881 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:13.351885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:13.351887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:13.352217 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:13.352538 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:13.352846 (+ 308us) server_negotiation.cc:299] Negotiation successful
0619 14:57:13.353085 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260619 14:57:13.354074 19250 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:13.454361 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 19122
I20260619 14:57:13.472798 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 19268
I20260619 14:57:13.487449 19488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.473777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55258 (local address 127.11.175.195:44397)
0619 14:57:13.473960 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:13.473965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:13.473984 (+ 19us) server_negotiation.cc:407] Connection header received
0619 14:57:13.474133 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:13.474137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:13.474200 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:13.474297 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:13.475948 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:13.476713 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:13.477933 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:13.478317 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:13.478415 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:13.480983 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:13.480999 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:13.481015 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:13.481064 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:13.484297 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:13.484948 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:13.484955 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:13.484957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:13.485095 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:13.485416 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:13.485420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:13.485422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:13.485792 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:13.486146 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:13.486476 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 14:57:13.486887 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 14:57:13.488363 19392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:13.489331 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.488394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:40446)
0619 14:57:13.488943 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.489072 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":252,"threads_started":1}
I20260619 14:57:13.491030 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:13.490464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45535 (local address 127.0.0.1:40456)
0619 14:57:13.490602 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:13.490874 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:45535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 14:57:13.609529 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 19268
I20260619 14:57:13.632216 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 18605
I20260619 14:57:13.633288 18773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:13.755791 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 18605
I20260619 14:57:13.777431 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 18707
I20260619 14:57:13.778582 18961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:13.905267 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 18707
2026-06-19T14:57:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9902 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:57:14 dist-test-slave-76rk krb5kdc[19528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:57:14 dist-test-slave-76rk krb5kdc[19528](info): set up 2 sockets
Jun 19 14:57:14 dist-test-slave-76rk krb5kdc[19528](info): commencing operation
krb5kdc: starting...
W20260619 14:57:15.032130 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:57:15 dist-test-slave-76rk krb5kdc[19528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:57:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:57:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.254.keytab.
Entry for principal oryx/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.254.keytab.
I20260619 14:57:15.270514 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41139
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.254.keytab
--principal=oryx/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:40905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:41139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:15.609483 19541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:15.609875 19541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:15.610055 19541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:15.617874 19541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:15.618049 19541 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 14:57:15.618151 19541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:15.618274 19541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:15.618364 19541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:15.618463 19541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:15.629169 19541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:41139
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.254.keytab
--principal=oryx/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:15.630595 19541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:15.632164 19541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:15.644667 19547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:15.644750 19546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:15.644809 19549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:15.645550 19541 server_base.cc:1034] running on GCE node
I20260619 14:57:15.646440 19541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:15.647949 19541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:15.649180 19541 hybrid_clock.cc:648] HybridClock initialized: now 1781881035649120 us; error 39 us; skew 500 ppm
I20260619 14:57:15.649551 19541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:15 dist-test-slave-76rk krb5kdc[19528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:15.652896 19541 init.cc:377] Logged in from keytab as oryx/127.11.175.254@KRBTEST.COM (short username oryx)
I20260619 14:57:15.654537 19541 webserver.cc:460] Webserver started at http://127.11.175.254:32963/ using document root <none> and password file <none>
I20260619 14:57:15.655318 19541 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:15.655408 19541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:15.655714 19541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:15.658596 19541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "821c6429344e4ec6aad943f1ea5b03ef"
format_stamp: "Formatted at 2026-06-19 14:57:15 on dist-test-slave-76rk"
server_key: "d14852f9c62deee92dbc8ee1c58e5593"
server_key_iv: "eb1362432b97f7fa10183b3ec6f2ea72"
server_key_version: "encryptionkey@0"
I20260619 14:57:15.659430 19541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "821c6429344e4ec6aad943f1ea5b03ef"
format_stamp: "Formatted at 2026-06-19 14:57:15 on dist-test-slave-76rk"
server_key: "d14852f9c62deee92dbc8ee1c58e5593"
server_key_iv: "eb1362432b97f7fa10183b3ec6f2ea72"
server_key_version: "encryptionkey@0"
I20260619 14:57:15.664445 19541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:57:15.668076 19556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:15.669533 19541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:57:15.669677 19541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "821c6429344e4ec6aad943f1ea5b03ef"
format_stamp: "Formatted at 2026-06-19 14:57:15 on dist-test-slave-76rk"
server_key: "d14852f9c62deee92dbc8ee1c58e5593"
server_key_iv: "eb1362432b97f7fa10183b3ec6f2ea72"
server_key_version: "encryptionkey@0"
I20260619 14:57:15.669864 19541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:15.702162 19541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:15.706871 19541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:15.707118 19541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:15.731235 19541 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:41139
I20260619 14:57:15.731215 19617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:41139 every 8 connection(s)
I20260619 14:57:15.732720 19541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:57:15.734328 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 19541
I20260619 14:57:15.734570 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:57:15.735261 11967 external_mini_cluster.cc:1428] Setting key fb6278d3ec07c4c30796a4cbefa47fb9
I20260619 14:57:15.737164 19618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:15.748776 19618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef: Bootstrap starting.
Jun 19 14:57:15 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:15.752091 19618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:15.753197 19618 log.cc:824] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:15.756696 19618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef: No bootstrap required, opened a new log
I20260619 14:57:15.758344 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:15.737371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.11.175.254:41139)
0619 14:57:15.738165 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:15.738177 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:15.738218 (+ 41us) server_negotiation.cc:407] Connection header received
0619 14:57:15.739936 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:15.739959 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:15.740406 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:15.740858 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:15.742325 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:15.743763 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:15.745017 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:15.745708 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:15.745950 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:15.750035 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:15.750044 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:15.750059 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:15.750104 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:15.753447 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:15.754348 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:15.754354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:15.754360 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:15.754618 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:15.755063 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:15.755071 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:15.755074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:15.755806 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:15.756090 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:15.756531 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 14:57:15.756975 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":360,"threads_started":1}
I20260619 14:57:15.765249 19618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821c6429344e4ec6aad943f1ea5b03ef" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41139 } }
I20260619 14:57:15.765702 19618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:15.765865 19618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821c6429344e4ec6aad943f1ea5b03ef, State: Initialized, Role: FOLLOWER
I20260619 14:57:15.766510 19618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821c6429344e4ec6aad943f1ea5b03ef" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41139 } }
I20260619 14:57:15.766813 19618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:57:15.766947 19618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:57:15.767164 19618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:15.769577 19618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821c6429344e4ec6aad943f1ea5b03ef" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41139 } }
I20260619 14:57:15.770186 19618 leader_election.cc:302] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 821c6429344e4ec6aad943f1ea5b03ef; no voters:
I20260619 14:57:15.770898 19618 leader_election.cc:288] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:57:15.771054 19623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:15.772019 19623 raft_consensus.cc:686] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [term 1 LEADER]: Becoming Leader. State: Replica: 821c6429344e4ec6aad943f1ea5b03ef, State: Running, Role: LEADER
I20260619 14:57:15.772614 19623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821c6429344e4ec6aad943f1ea5b03ef" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41139 } }
I20260619 14:57:15.773394 19618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:57:15.775763 19625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 821c6429344e4ec6aad943f1ea5b03ef. Latest consensus state: current_term: 1 leader_uuid: "821c6429344e4ec6aad943f1ea5b03ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821c6429344e4ec6aad943f1ea5b03ef" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41139 } } }
I20260619 14:57:15.776216 19625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:15.777122 19624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "821c6429344e4ec6aad943f1ea5b03ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821c6429344e4ec6aad943f1ea5b03ef" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41139 } } }
I20260619 14:57:15.777508 19624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:15.779671 19628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:57:15.785446 19628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:57:15.794711 19628 catalog_manager.cc:1347] Generated new cluster ID: a2a4c501b17c46099e1e386ec28cf220
I20260619 14:57:15.794895 19628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:57:15.819025 19628 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:57:15.820395 19628 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:57:15.829092 19628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef: Generated new TSK 0
I20260619 14:57:15.829969 19628 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab.
Entry for principal oryx/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab.
I20260619 14:57:15.910331 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41139
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab
--principal=oryx/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:40905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:16.259943 19644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:16.260241 19644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:16.260416 19644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:16.268077 19644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:16.268199 19644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:16.268352 19644 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 14:57:16.268471 19644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:57:16.279219 19644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab
--principal=oryx/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.175.254:41139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:16.280653 19644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:16.282023 19644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:16.296283 19649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:16.296314 19652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:16.296353 19650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:16.297219 19644 server_base.cc:1034] running on GCE node
I20260619 14:57:16.297814 19644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:16.298889 19644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:16.300165 19644 hybrid_clock.cc:648] HybridClock initialized: now 1781881036300068 us; error 63 us; skew 500 ppm
I20260619 14:57:16.300521 19644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:16 dist-test-slave-76rk krb5kdc[19528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881036, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:16.303954 19644 init.cc:377] Logged in from keytab as oryx/127.11.175.193@KRBTEST.COM (short username oryx)
I20260619 14:57:16.305470 19644 webserver.cc:460] Webserver started at http://127.11.175.193:43897/ using document root <none> and password file <none>
I20260619 14:57:16.306180 19644 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:16.306268 19644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:16.306540 19644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:16.309548 19644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "e688dc32f58f4c65829fe056b3a17f2d"
format_stamp: "Formatted at 2026-06-19 14:57:16 on dist-test-slave-76rk"
server_key: "d711a4d7405dfbecbd0c05afc20f9112"
server_key_iv: "004aabad6e2b5280b0055ce10f656ef2"
server_key_version: "encryptionkey@0"
I20260619 14:57:16.310364 19644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "e688dc32f58f4c65829fe056b3a17f2d"
format_stamp: "Formatted at 2026-06-19 14:57:16 on dist-test-slave-76rk"
server_key: "d711a4d7405dfbecbd0c05afc20f9112"
server_key_iv: "004aabad6e2b5280b0055ce10f656ef2"
server_key_version: "encryptionkey@0"
I20260619 14:57:16.315948 19644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 14:57:16.319792 19659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:16.321445 19644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:57:16.321666 19644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "e688dc32f58f4c65829fe056b3a17f2d"
format_stamp: "Formatted at 2026-06-19 14:57:16 on dist-test-slave-76rk"
server_key: "d711a4d7405dfbecbd0c05afc20f9112"
server_key_iv: "004aabad6e2b5280b0055ce10f656ef2"
server_key_version: "encryptionkey@0"
I20260619 14:57:16.321933 19644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:16.344661 19644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:16.349313 19644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:16.349632 19644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:16.352591 19644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:16.352705 19644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:16.352883 19644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:16.352960 19644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 14:57:16 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881036, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:16.379081 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:16.353054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:47219 (local address 127.11.175.254:41139)
0619 14:57:16.353872 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:16.353878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:16.355035 (+ 1157us) server_negotiation.cc:407] Connection header received
0619 14:57:16.357210 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:16.357217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:16.357303 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:16.357439 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:16.359820 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.361142 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.363819 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.364477 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.364616 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:16.370765 (+ 6149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:16.370775 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:16.370783 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:16.370836 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:16.375154 (+ 4318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:16.376051 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:16.376060 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:16.376066 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:16.376269 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:16.376747 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:16.376755 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:16.376761 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:16.377228 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:16.377409 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:16.378409 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0619 14:57:16.378712 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":575,"threads_started":1}
I20260619 14:57:16.380450 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:16.353834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41139 (local address 127.11.175.193:47219)
0619 14:57:16.354843 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:16.354893 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:16.356856 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:16.357709 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:16.357728 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:16.358609 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:16.359556 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:16.359591 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.361352 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.361360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:16.363659 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:16.363673 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.364761 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.364770 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:16.364980 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:16.365779 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:16.365828 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:16.370372 (+ 4544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:16.375345 (+ 4973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:16.375353 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:16.375370 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:16.375891 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:16.376392 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:16.376395 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:16.376397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:16.376614 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:16.377374 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:16.377380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:16.377974 (+ 594us) client_negotiation.cc:769] Sending connection context
0619 14:57:16.378303 (+ 329us) client_negotiation.cc:240] Negotiation successful
0619 14:57:16.378959 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":412,"threads_started":1}
I20260619 14:57:16.403007 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:16.387153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:39035 (local address 127.11.175.254:41139)
0619 14:57:16.387338 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:16.387343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:16.387995 (+ 652us) server_negotiation.cc:407] Connection header received
0619 14:57:16.388217 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:16.388223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:16.388327 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:16.388434 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:16.390071 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.391233 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.393413 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.393872 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.394014 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:16.396317 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:16.396324 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:16.396329 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:16.396411 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:16.400150 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:16.400892 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:16.400899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:16.400903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:16.401049 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:16.401440 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:16.401446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:16.401449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:16.401752 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:16.401921 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:16.402333 (+ 412us) server_negotiation.cc:299] Negotiation successful
0619 14:57:16.402580 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 14:57:16.403294 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:16.387235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41139 (local address 127.11.175.193:39035)
0619 14:57:16.387861 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:16.387879 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:16.388075 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:16.388639 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:16.388643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:16.389372 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:16.389876 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:16.389901 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.391379 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.391388 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:16.393250 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:16.393263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.394089 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.394097 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:16.394248 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:16.394975 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:16.394997 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:16.395932 (+ 935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:16.400309 (+ 4377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:16.400315 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:16.400319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:16.400755 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:16.401172 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:16.401177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:16.401181 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:16.401322 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:16.401879 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:16.401884 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:16.402081 (+ 197us) client_negotiation.cc:769] Sending connection context
0619 14:57:16.402231 (+ 150us) client_negotiation.cc:240] Negotiation successful
0619 14:57:16.402592 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":356,"threads_started":1}
I20260619 14:57:16.407696 19644 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:34063
I20260619 14:57:16.407785 19779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:34063 every 8 connection(s)
I20260619 14:57:16.409097 19644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:57:16.411459 19780 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41139
I20260619 14:57:16.411720 19780 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:16.412441 19780 heartbeater.cc:511] Master 127.11.175.254:41139 requested a full tablet report, sending...
I20260619 14:57:16.414531 19573 ts_manager.cc:194] Registered new tserver with Master: e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063)
I20260619 14:57:16.415858 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 19644
I20260619 14:57:16.416049 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:57:16.416249 19573 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:47219
I20260619 14:57:16.416774 11967 external_mini_cluster.cc:1428] Setting key fd3b8efd6a77d1c697262f85e825bb38
WARNING: no policy specified for oryx/127.11.175.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab.
Entry for principal oryx/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab.
I20260619 14:57:16.467020 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41139
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab
--principal=oryx/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:40905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:16.807303 19784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:16.807677 19784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:16.807866 19784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:16.815274 19784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:16.815461 19784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:16.815650 19784 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 14:57:16.815810 19784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:57:16.826597 19784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab
--principal=oryx/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.175.254:41139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:16.827997 19784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:16.829350 19784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:16.843688 19789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:16.843650 19790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:16.843639 19792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:16.844974 19784 server_base.cc:1034] running on GCE node
I20260619 14:57:16.845676 19784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:16.846762 19784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:16.848029 19784 hybrid_clock.cc:648] HybridClock initialized: now 1781881036847965 us; error 38 us; skew 500 ppm
I20260619 14:57:16.848376 19784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:16 dist-test-slave-76rk krb5kdc[19528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881036, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:16.851863 19784 init.cc:377] Logged in from keytab as oryx/127.11.175.194@KRBTEST.COM (short username oryx)
I20260619 14:57:16.853606 19784 webserver.cc:460] Webserver started at http://127.11.175.194:42341/ using document root <none> and password file <none>
I20260619 14:57:16.854338 19784 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:16.854475 19784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:16.854864 19784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:16.857908 19784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "b53d293281434901ad4bbe5b453b8942"
format_stamp: "Formatted at 2026-06-19 14:57:16 on dist-test-slave-76rk"
server_key: "4b8675c656b12c7d2c567b4253cb3caa"
server_key_iv: "174f8a94ba030bb1dbc5722c7dd3871a"
server_key_version: "encryptionkey@0"
I20260619 14:57:16.858775 19784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "b53d293281434901ad4bbe5b453b8942"
format_stamp: "Formatted at 2026-06-19 14:57:16 on dist-test-slave-76rk"
server_key: "4b8675c656b12c7d2c567b4253cb3caa"
server_key_iv: "174f8a94ba030bb1dbc5722c7dd3871a"
server_key_version: "encryptionkey@0"
I20260619 14:57:16.864578 19784 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 14:57:16.868938 19799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:16.870558 19784 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 14:57:16.870828 19784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "b53d293281434901ad4bbe5b453b8942"
format_stamp: "Formatted at 2026-06-19 14:57:16 on dist-test-slave-76rk"
server_key: "4b8675c656b12c7d2c567b4253cb3caa"
server_key_iv: "174f8a94ba030bb1dbc5722c7dd3871a"
server_key_version: "encryptionkey@0"
I20260619 14:57:16.871068 19784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:16.897262 19784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:16.901772 19784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:16.902086 19784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:16.905099 19784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:16.905232 19784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:16.905411 19784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:16.905493 19784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 14:57:16 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881036, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:16.931460 19808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:16.906237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:36061 (local address 127.11.175.254:41139)
0619 14:57:16.906794 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:16.906799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:16.908174 (+ 1375us) server_negotiation.cc:407] Connection header received
0619 14:57:16.910249 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:16.910256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:16.910352 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:16.910484 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:16.913180 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.914334 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.917077 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.917671 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.917813 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:16.923698 (+ 5885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:16.923705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:16.923713 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:16.923761 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:16.927553 (+ 3792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:16.928821 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:16.928829 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:16.928832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:16.928988 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:16.929503 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:16.929508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:16.929511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:16.929818 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:16.930009 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:16.930822 (+ 813us) server_negotiation.cc:299] Negotiation successful
0619 14:57:16.931072 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":308,"threads_started":1}
I20260619 14:57:16.932986 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:16.907092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41139 (local address 127.11.175.194:36061)
0619 14:57:16.907986 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:16.908027 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:16.909943 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:16.910952 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:16.910964 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:16.911877 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:16.912932 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:16.912979 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.914520 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.914527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:16.916906 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:16.916921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.918167 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.918179 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:16.918469 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:16.919491 (+ 1022us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:16.919528 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:16.923331 (+ 3803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:16.927748 (+ 4417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:16.927759 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:16.927782 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:16.928615 (+ 833us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:16.929118 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:16.929122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:16.929126 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:16.929357 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:16.929968 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:16.929978 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:16.930384 (+ 406us) client_negotiation.cc:769] Sending connection context
0619 14:57:16.930695 (+ 311us) client_negotiation.cc:240] Negotiation successful
0619 14:57:16.931313 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":379,"threads_started":1}
I20260619 14:57:16.955236 19784 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:43983
I20260619 14:57:16.955258 19919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:43983 every 8 connection(s)
I20260619 14:57:16.956604 19784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:57:16.958619 19887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:16.940704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41139 (local address 127.11.175.194:35107)
0619 14:57:16.941431 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:16.941448 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:16.941634 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:16.942031 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:16.942035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:16.942566 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:16.943027 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:16.943040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.944799 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.944808 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:16.946771 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:16.946796 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.948219 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.948226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:16.948476 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:16.949434 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:16.949468 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:16.950989 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:16.955220 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:16.955227 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:16.955231 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:16.955868 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:16.956335 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:16.956341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:16.956345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:16.956503 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:16.957278 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:16.957287 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:16.957555 (+ 268us) client_negotiation.cc:769] Sending connection context
0619 14:57:16.957743 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 14:57:16.958137 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":442,"threads_started":1}
I20260619 14:57:16.959183 19920 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41139
I20260619 14:57:16.959329 19808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:16.940670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:35107 (local address 127.11.175.254:41139)
0619 14:57:16.940889 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:16.940897 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:16.941557 (+ 660us) server_negotiation.cc:407] Connection header received
0619 14:57:16.941790 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:16.941796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:16.941874 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:16.942006 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:16.943256 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.944528 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.946969 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:16.947646 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:16.947809 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:16.951211 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:16.951218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:16.951223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:16.951271 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:16.955064 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:16.956028 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:16.956034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:16.956039 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:16.956198 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:16.956642 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:16.956647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:16.956651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:16.957114 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:16.957309 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:16.958876 (+ 1567us) server_negotiation.cc:299] Negotiation successful
0619 14:57:16.959094 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 14:57:16.959460 19920 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:16.960282 19920 heartbeater.cc:511] Master 127.11.175.254:41139 requested a full tablet report, sending...
I20260619 14:57:16.961843 19573 ts_manager.cc:194] Registered new tserver with Master: b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983)
I20260619 14:57:16.962968 19573 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:36061
I20260619 14:57:16.965420 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 19784
I20260619 14:57:16.965603 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:57:16.966025 11967 external_mini_cluster.cc:1428] Setting key 61ac5fec7c9b0657067c516879e11680
WARNING: no policy specified for oryx/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab.
Entry for principal oryx/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab.
I20260619 14:57:17.019613 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41139
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab
--principal=oryx/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:40905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:17.362986 19924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:17.363365 19924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:17.363565 19924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:17.371434 19924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:17.371630 19924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:17.371793 19924 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 14:57:17.371965 19924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:57:17.383366 19924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab
--principal=oryx/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.175.254:41139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:17.384891 19924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:17.386369 19924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:17.401114 19932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:17.401130 19930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:17.401181 19929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:17.402159 19924 server_base.cc:1034] running on GCE node
I20260619 14:57:17.403005 19924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:17.404174 19924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:17.405476 19924 hybrid_clock.cc:648] HybridClock initialized: now 1781881037405406 us; error 44 us; skew 500 ppm
I20260619 14:57:17.405848 19924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:17 dist-test-slave-76rk krb5kdc[19528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881037, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:17.409605 19924 init.cc:377] Logged in from keytab as oryx/127.11.175.195@KRBTEST.COM (short username oryx)
I20260619 14:57:17.411373 19924 webserver.cc:460] Webserver started at http://127.11.175.195:45131/ using document root <none> and password file <none>
I20260619 14:57:17.412137 19924 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:17.412276 19924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:17.412608 19924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:17.415813 19924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "cae97e7c1e0a45708ebb4e4bd08f6254"
format_stamp: "Formatted at 2026-06-19 14:57:17 on dist-test-slave-76rk"
server_key: "184bee7c294f48c50e2c9c776c16d9e1"
server_key_iv: "4f291a3aab49e8b1a61b560c3fd096a0"
server_key_version: "encryptionkey@0"
I20260619 14:57:17.416680 19924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "cae97e7c1e0a45708ebb4e4bd08f6254"
format_stamp: "Formatted at 2026-06-19 14:57:17 on dist-test-slave-76rk"
server_key: "184bee7c294f48c50e2c9c776c16d9e1"
server_key_iv: "4f291a3aab49e8b1a61b560c3fd096a0"
server_key_version: "encryptionkey@0"
I20260619 14:57:17.419670 19780 heartbeater.cc:503] Master 127.11.175.254:41139 was elected leader, sending a full tablet report...
I20260619 14:57:17.423348 19924 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 14:57:17.427140 19939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:17.428654 19924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 14:57:17.428872 19924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "cae97e7c1e0a45708ebb4e4bd08f6254"
format_stamp: "Formatted at 2026-06-19 14:57:17 on dist-test-slave-76rk"
server_key: "184bee7c294f48c50e2c9c776c16d9e1"
server_key_iv: "4f291a3aab49e8b1a61b560c3fd096a0"
server_key_version: "encryptionkey@0"
I20260619 14:57:17.429138 19924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:17.464643 19924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:17.469630 19924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:17.469981 19924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:17.473204 19924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:17.473325 19924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:17.473475 19924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:17.473567 19924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 14:57:17 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881037, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:17.501804 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.473982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:56611 (local address 127.11.175.254:41139)
0619 14:57:17.474760 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.474768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.476087 (+ 1319us) server_negotiation.cc:407] Connection header received
0619 14:57:17.478218 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.478224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.478311 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.478452 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:17.481140 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.482205 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.485080 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.485765 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.485868 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.492931 (+ 7063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:17.492939 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:17.492944 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:17.493070 (+ 126us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:17.496811 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:17.498073 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:17.498080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:17.498084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:17.498280 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:17.499039 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:17.499048 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:17.499054 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:17.499555 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:17.499750 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.500940 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.501242 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":466,"threads_started":1}
I20260619 14:57:17.503499 19949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.474778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41139 (local address 127.11.175.195:56611)
0619 14:57:17.475896 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:17.475936 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:17.477916 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:17.478815 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:17.478823 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:17.479710 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:17.480936 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.480957 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.482430 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.482441 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.484823 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.484866 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.486098 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.486107 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.486552 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.487627 (+ 1075us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:17.487674 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:17.492555 (+ 4881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:17.496993 (+ 4438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:17.497010 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:17.497034 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:17.497861 (+ 827us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:17.498473 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:17.498483 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:17.498486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:17.498849 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:17.499724 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:17.499737 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:17.500371 (+ 634us) client_negotiation.cc:769] Sending connection context
0619 14:57:17.500754 (+ 383us) client_negotiation.cc:240] Negotiation successful
0619 14:57:17.501500 (+ 746us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":397,"threads_started":1}
I20260619 14:57:17.520324 20059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:38527 every 8 connection(s)
I20260619 14:57:17.520370 19924 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:38527
I20260619 14:57:17.522281 19924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:57:17.524578 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 19924
I20260619 14:57:17.524760 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:57:17.525193 11967 external_mini_cluster.cc:1428] Setting key 3261c456036562ef2406b65d463cf3cb
I20260619 14:57:17.527237 20060 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41139
I20260619 14:57:17.527506 20060 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:17.528328 20060 heartbeater.cc:511] Master 127.11.175.254:41139 requested a full tablet report, sending...
I20260619 14:57:17.529502 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.511275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41139 (local address 127.11.175.195:54469)
0619 14:57:17.511996 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:17.512016 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:17.512145 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:17.512805 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:17.512813 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:17.513692 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:17.514351 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.514371 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.515910 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.515919 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.518305 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.518321 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.519242 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.519253 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.519540 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.520434 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:17.520462 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:17.521608 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:17.525857 (+ 4249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:17.525862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:17.525866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:17.526542 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:17.527102 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:17.527107 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:17.527111 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:17.527291 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:17.528056 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:17.528061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:17.528676 (+ 615us) client_negotiation.cc:769] Sending connection context
0619 14:57:17.528798 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 14:57:17.529051 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":391,"threads_started":1}
I20260619 14:57:17.529711 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.511206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:54469 (local address 127.11.175.254:41139)
0619 14:57:17.511431 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.511439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.512131 (+ 692us) server_negotiation.cc:407] Connection header received
0619 14:57:17.512327 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.512335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.512433 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.512569 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:17.514576 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.515707 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.518468 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.518993 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.519147 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.521881 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:17.521889 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:17.521894 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:17.521939 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:17.525646 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:17.526721 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:17.526727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:17.526730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:17.526921 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:17.527448 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:17.527455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:17.527458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:17.527890 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:17.528060 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.528994 (+ 934us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.529234 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 14:57:17.529965 19573 ts_manager.cc:194] Registered new tserver with Master: cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527)
I20260619 14:57:17.530845 19573 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:56611
I20260619 14:57:17.540751 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 14:57:17 dist-test-slave-76rk krb5kdc[19528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881037, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 14:57:17 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:17 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 14:57:17.594601 19948 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:17.583449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43376 (local address 127.11.175.254:41139)
0619 14:57:17.583656 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.583660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.584166 (+ 506us) server_negotiation.cc:407] Connection header received
0619 14:57:17.584315 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.584320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.584391 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.584482 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:17.585829 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.586563 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.587857 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.588276 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.588374 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.594227 (+ 5853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43376: BlockingRecv error: recv got EOF from 127.0.0.1:43376 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 14:57:17 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881037, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:17.611444 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.597833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43392 (local address 127.11.175.254:41139)
0619 14:57:17.598042 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.598047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.598435 (+ 388us) server_negotiation.cc:407] Connection header received
0619 14:57:17.598553 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.598557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.598622 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.598787 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:17.599951 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.600674 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.601893 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.602337 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.602453 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.606292 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:17.606297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:17.606301 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:17.606335 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:17.609024 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:17.609671 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:17.609675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:17.609678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:17.609820 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:17.610185 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:17.610189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:17.610191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:17.610470 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:17.610745 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.611032 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.611207 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260619 14:57:17.615541 19572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:57:17.622782 19572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:57:17.647679 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.635889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:43983 (local address 127.0.0.1:53872)
0619 14:57:17.636833 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:17.636853 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:17.636986 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:17.639155 (+ 2169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:17.639165 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:17.639208 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:17.639668 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.639682 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.641985 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.641992 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.644492 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.644521 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.646513 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.646520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.646768 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.646837 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 14:57:17.646958 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 14:57:17.647215 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":96,"thread_start_us":1116,"threads_started":1}
I20260619 14:57:17.648263 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.637081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.11.175.193:34063)
0619 14:57:17.637572 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.637581 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.640210 (+ 2629us) server_negotiation.cc:407] Connection header received
0619 14:57:17.640422 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.640432 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.640706 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.640901 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:17.641848 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.643489 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.645597 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.646686 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.646861 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.647036 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.647785 (+ 749us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.647974 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":468,"threads_started":1}
I20260619 14:57:17.648363 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.636429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53872 (local address 127.11.175.194:43983)
0619 14:57:17.637709 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.637719 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.637753 (+ 34us) server_negotiation.cc:407] Connection header received
0619 14:57:17.637889 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.637902 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.638255 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.638537 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:17.639856 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.641821 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.644722 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.646278 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.646472 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.647417 (+ 945us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.647500 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.647685 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1111,"thread_start_us":1117,"threads_started":1}
I20260619 14:57:17.648823 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.636885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:34063 (local address 127.0.0.1:60102)
0619 14:57:17.640079 (+ 3194us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:17.640094 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:17.640239 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:17.641134 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:17.641141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:17.641170 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:17.641674 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.641684 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.643651 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.643657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.645422 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.645433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.647197 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.647205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.647378 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.647411 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 14:57:17.647500 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 14:57:17.647673 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3089,"mutex_wait_us":27,"thread_start_us":2974,"threads_started":1}
I20260619 14:57:17.650386 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.635888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38527 (local address 127.0.0.1:49010)
0619 14:57:17.636746 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:17.636787 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:17.636984 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:17.641235 (+ 4251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:17.641241 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:17.641276 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:17.641853 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.641869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.644345 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.644352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.646674 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.646690 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.649580 (+ 2890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.649587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.649761 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.649801 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 14:57:17.649896 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 14:57:17.650036 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":4,"thread_start_us":1350,"threads_started":1}
I20260619 14:57:17.650715 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.636518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49010 (local address 127.11.175.195:38527)
0619 14:57:17.640629 (+ 4111us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.640637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.640668 (+ 31us) server_negotiation.cc:407] Connection header received
0619 14:57:17.640772 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.640779 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.641003 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.641210 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:17.642079 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.644136 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.646891 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.649356 (+ 2465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.649585 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.649722 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.650113 (+ 391us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.650287 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3948,"thread_start_us":3079,"threads_started":1}
I20260619 14:57:17.652707 19850 tablet_service.cc:1467] Processing CreateTablet for tablet 687c0a54a710471db9fb12ac9244079b (DEFAULT_TABLE table=test-table [id=15c6528a2b3143d4a6461c40e5aac431]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:17.652737 19710 tablet_service.cc:1467] Processing CreateTablet for tablet 687c0a54a710471db9fb12ac9244079b (DEFAULT_TABLE table=test-table [id=15c6528a2b3143d4a6461c40e5aac431]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:17.653954 19850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 687c0a54a710471db9fb12ac9244079b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:17.654158 19710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 687c0a54a710471db9fb12ac9244079b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:17.655238 19990 tablet_service.cc:1467] Processing CreateTablet for tablet 687c0a54a710471db9fb12ac9244079b (DEFAULT_TABLE table=test-table [id=15c6528a2b3143d4a6461c40e5aac431]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:17.657405 19990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 687c0a54a710471db9fb12ac9244079b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:17.665531 20084 tablet_bootstrap.cc:492] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: Bootstrap starting.
I20260619 14:57:17.668617 20084 tablet_bootstrap.cc:654] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:17.669587 20085 tablet_bootstrap.cc:492] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: Bootstrap starting.
I20260619 14:57:17.670377 20084 log.cc:824] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:17.673408 20085 tablet_bootstrap.cc:654] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:17.673434 20087 tablet_bootstrap.cc:492] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: Bootstrap starting.
I20260619 14:57:17.674928 20084 tablet_bootstrap.cc:492] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: No bootstrap required, opened a new log
I20260619 14:57:17.675019 20085 log.cc:824] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:17.675298 20084 ts_tablet_manager.cc:1397] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260619 14:57:17.677268 20087 tablet_bootstrap.cc:654] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:17.678349 20085 tablet_bootstrap.cc:492] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: No bootstrap required, opened a new log
I20260619 14:57:17.678342 20087 log.cc:824] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:17.678620 20085 ts_tablet_manager.cc:1397] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260619 14:57:17.683305 20087 tablet_bootstrap.cc:492] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: No bootstrap required, opened a new log
I20260619 14:57:17.683624 20087 ts_tablet_manager.cc:1397] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260619 14:57:17.686048 20084 raft_consensus.cc:348] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } }
I20260619 14:57:17.686507 20084 raft_consensus.cc:374] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:17.686597 20084 raft_consensus.cc:729] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b53d293281434901ad4bbe5b453b8942, State: Initialized, Role: FOLLOWER
I20260619 14:57:17.687260 20084 consensus_queue.cc:260] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } }
I20260619 14:57:17.689947 20085 raft_consensus.cc:348] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } }
I20260619 14:57:17.690518 20085 raft_consensus.cc:374] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:17.690654 20085 raft_consensus.cc:729] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e688dc32f58f4c65829fe056b3a17f2d, State: Initialized, Role: FOLLOWER
I20260619 14:57:17.691568 20085 consensus_queue.cc:260] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } }
I20260619 14:57:17.692240 20087 raft_consensus.cc:348] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } }
I20260619 14:57:17.692725 20087 raft_consensus.cc:374] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:17.692852 20087 raft_consensus.cc:729] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae97e7c1e0a45708ebb4e4bd08f6254, State: Initialized, Role: FOLLOWER
I20260619 14:57:17.693491 19920 heartbeater.cc:503] Master 127.11.175.254:41139 was elected leader, sending a full tablet report...
I20260619 14:57:17.693609 20087 consensus_queue.cc:260] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } }
I20260619 14:57:17.694350 20084 ts_tablet_manager.cc:1428] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: Time spent starting tablet: real 0.019s user 0.018s sys 0.000s
I20260619 14:57:17.696132 20085 ts_tablet_manager.cc:1428] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: Time spent starting tablet: real 0.017s user 0.007s sys 0.010s
I20260619 14:57:17.696862 20090 raft_consensus.cc:482] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:57:17.697190 20090 raft_consensus.cc:504] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } }
I20260619 14:57:17.697412 20060 heartbeater.cc:503] Master 127.11.175.254:41139 was elected leader, sending a full tablet report...
I20260619 14:57:17.699113 20087 ts_tablet_manager.cc:1428] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260619 14:57:17.701207 20090 leader_election.cc:288] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527), e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063)
W20260619 14:57:17.708802 19921 tablet.cc:2369] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:57:17.711082 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.700863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:42141 (local address 127.11.175.195:38527)
0619 14:57:17.701085 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.701093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.704213 (+ 3120us) server_negotiation.cc:407] Connection header received
0619 14:57:17.704481 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.704486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.704575 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.704637 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:17.705427 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.706452 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.708785 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.709965 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.710124 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.710193 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.710692 (+ 499us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.710826 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 14:57:17.711069 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.700154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:44137 (local address 127.11.175.193:34063)
0619 14:57:17.700405 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.700414 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.703672 (+ 3258us) server_negotiation.cc:407] Connection header received
0619 14:57:17.704672 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.704679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.704776 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.704837 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:17.706015 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.706865 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.708687 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.709883 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.710067 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.710144 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.710697 (+ 553us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.710826 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 14:57:17.711489 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.700571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38527 (local address 127.11.175.194:42141)
0619 14:57:17.704126 (+ 3555us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:17.704143 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:17.704320 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:17.704834 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:17.704838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:17.704860 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:17.705243 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.705255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.706609 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.706618 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.708635 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.708652 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.710214 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.710220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.710449 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.710520 (+ 71us) client_negotiation.cc:769] Sending connection context
0619 14:57:17.710653 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 14:57:17.710947 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3441,"mutex_wait_us":105,"thread_start_us":2530,"threads_started":1}
I20260619 14:57:17.711516 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.699562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:34063 (local address 127.11.175.194:44137)
0619 14:57:17.703549 (+ 3987us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:17.703575 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:17.704521 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:17.705029 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:17.705035 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:17.705069 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:17.705865 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.705877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.706995 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.707001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.708513 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.708525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.710210 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.710217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.710506 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.710535 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 14:57:17.710656 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 14:57:17.710946 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3795,"mutex_wait_us":579,"thread_start_us":4284,"threads_started":1}
I20260619 14:57:17.712549 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687c0a54a710471db9fb12ac9244079b" candidate_uuid: "b53d293281434901ad4bbe5b453b8942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e688dc32f58f4c65829fe056b3a17f2d" is_pre_election: true
I20260619 14:57:17.712555 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687c0a54a710471db9fb12ac9244079b" candidate_uuid: "b53d293281434901ad4bbe5b453b8942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" is_pre_election: true
I20260619 14:57:17.713043 19730 raft_consensus.cc:2454] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b53d293281434901ad4bbe5b453b8942 in term 0.
I20260619 14:57:17.713112 20010 raft_consensus.cc:2454] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b53d293281434901ad4bbe5b453b8942 in term 0.
I20260619 14:57:17.713716 19803 leader_election.cc:302] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b53d293281434901ad4bbe5b453b8942, e688dc32f58f4c65829fe056b3a17f2d; no voters:
I20260619 14:57:17.714146 20090 raft_consensus.cc:2781] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:57:17.714295 20090 raft_consensus.cc:482] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:57:17.714407 20090 raft_consensus.cc:3037] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:17.716524 20090 raft_consensus.cc:504] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } }
I20260619 14:57:17.717074 20090 leader_election.cc:288] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [CANDIDATE]: Term 1 election: Requested vote from peers cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527), e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063)
I20260619 14:57:17.717507 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687c0a54a710471db9fb12ac9244079b" candidate_uuid: "b53d293281434901ad4bbe5b453b8942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254"
I20260619 14:57:17.717530 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687c0a54a710471db9fb12ac9244079b" candidate_uuid: "b53d293281434901ad4bbe5b453b8942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e688dc32f58f4c65829fe056b3a17f2d"
I20260619 14:57:17.717734 19730 raft_consensus.cc:3037] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:17.717727 20010 raft_consensus.cc:3037] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:17.719977 19730 raft_consensus.cc:2454] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b53d293281434901ad4bbe5b453b8942 in term 1.
I20260619 14:57:17.720047 20010 raft_consensus.cc:2454] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b53d293281434901ad4bbe5b453b8942 in term 1.
I20260619 14:57:17.720402 19803 leader_election.cc:302] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b53d293281434901ad4bbe5b453b8942, e688dc32f58f4c65829fe056b3a17f2d; no voters:
I20260619 14:57:17.720697 20090 raft_consensus.cc:2781] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:17.721196 20090 raft_consensus.cc:686] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 1 LEADER]: Becoming Leader. State: Replica: b53d293281434901ad4bbe5b453b8942, State: Running, Role: LEADER
I20260619 14:57:17.721771 20090 consensus_queue.cc:237] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } }
I20260619 14:57:17.726544 19572 catalog_manager.cc:5495] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 reported cstate change: term changed from 0 to 1, leader changed from <none> to b53d293281434901ad4bbe5b453b8942 (127.11.175.194). New cstate: current_term: 1 leader_uuid: "b53d293281434901ad4bbe5b453b8942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } health_report { overall_health: UNKNOWN } } }
I20260619 14:57:17.755107 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.749392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.11.175.194:43983)
0619 14:57:17.749637 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.749646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.749743 (+ 97us) server_negotiation.cc:407] Connection header received
0619 14:57:17.750016 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.750020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.750100 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.750265 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:57:17.750666 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.751415 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.752812 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.753233 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.753381 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.753861 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:57:17.754426 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:57:17.754617 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.754736 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.754827 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 14:57:17.763798 20010 raft_consensus.cc:1261] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Refusing update from remote peer b53d293281434901ad4bbe5b453b8942: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:17.764046 19730 raft_consensus.cc:1261] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Refusing update from remote peer b53d293281434901ad4bbe5b453b8942: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:17.765002 20095 consensus_queue.cc:1035] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:17.765431 20090 consensus_queue.cc:1035] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 14:57:17.774948 20061 tablet.cc:2369] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:57:17.777271 20098 mvcc.cc:205] Tried to move back new op lower bound from 7298584730669330432 to 7298584730512986112. Current Snapshot: MvccSnapshot[applied={T|T < 7298584730669330432}]
I20260619 14:57:17.778112 20099 mvcc.cc:205] Tried to move back new op lower bound from 7298584730669330432 to 7298584730512986112. Current Snapshot: MvccSnapshot[applied={T|T < 7298584730669330432}]
I20260619 14:57:17.793207 19573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:17.793479 19573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:17.797833 19573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef: Sending DeleteTablet for 3 replicas of tablet 687c0a54a710471db9fb12ac9244079b
I20260619 14:57:17.799192 19850 tablet_service.cc:1514] Processing DeleteTablet for tablet 687c0a54a710471db9fb12ac9244079b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:17 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:53872
I20260619 14:57:17.799299 19990 tablet_service.cc:1514] Processing DeleteTablet for tablet 687c0a54a710471db9fb12ac9244079b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:17 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:49010
I20260619 14:57:17.800215 20108 tablet_replica.cc:330] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: stopping tablet replica
I20260619 14:57:17.800282 19710 tablet_service.cc:1514] Processing DeleteTablet for tablet 687c0a54a710471db9fb12ac9244079b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:17 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:60102
I20260619 14:57:17.800832 20108 raft_consensus.cc:2229] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:57:17.801589 20108 raft_consensus.cc:2258] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:17.801622 19573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:57:17.802416 19573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:57:17.809942 19850 tablet_service.cc:1467] Processing CreateTablet for tablet 619c1c5ab1404500ab0d463cd8e9a930 (DEFAULT_TABLE table=test-table [id=47a3cca872484bbdbab0150e5fc0383f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:17.810218 19989 tablet_service.cc:1467] Processing CreateTablet for tablet 619c1c5ab1404500ab0d463cd8e9a930 (DEFAULT_TABLE table=test-table [id=47a3cca872484bbdbab0150e5fc0383f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:17.810508 19850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 619c1c5ab1404500ab0d463cd8e9a930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:17.810894 19989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 619c1c5ab1404500ab0d463cd8e9a930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:17.811215 19709 tablet_service.cc:1467] Processing CreateTablet for tablet 619c1c5ab1404500ab0d463cd8e9a930 (DEFAULT_TABLE table=test-table [id=47a3cca872484bbdbab0150e5fc0383f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:17.811743 19709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 619c1c5ab1404500ab0d463cd8e9a930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:17.818733 20085 tablet_bootstrap.cc:492] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: Bootstrap starting.
I20260619 14:57:17.820176 20085 tablet_bootstrap.cc:654] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:17.823939 20087 tablet_bootstrap.cc:492] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254: Bootstrap starting.
I20260619 14:57:17.826107 20087 tablet_bootstrap.cc:654] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:17.827474 20084 tablet_bootstrap.cc:492] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942: Bootstrap starting.
I20260619 14:57:17.828316 20108 ts_tablet_manager.cc:1905] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:17.830000 20084 tablet_bootstrap.cc:654] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:17.834700 20084 tablet_bootstrap.cc:492] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942: No bootstrap required, opened a new log
I20260619 14:57:17.835062 20084 ts_tablet_manager.cc:1397] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260619 14:57:17.836349 20084 raft_consensus.cc:348] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:17.836813 20084 raft_consensus.cc:374] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:17.836992 20084 raft_consensus.cc:729] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b53d293281434901ad4bbe5b453b8942, State: Initialized, Role: FOLLOWER
I20260619 14:57:17.837342 20084 consensus_queue.cc:260] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:17.839563 20084 ts_tablet_manager.cc:1428] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260619 14:57:17.841868 20087 tablet_bootstrap.cc:492] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254: No bootstrap required, opened a new log
I20260619 14:57:17.842130 20087 ts_tablet_manager.cc:1397] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.007s
I20260619 14:57:17.843178 20087 raft_consensus.cc:348] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:17.843422 20087 raft_consensus.cc:374] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:17.843523 20087 raft_consensus.cc:729] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae97e7c1e0a45708ebb4e4bd08f6254, State: Initialized, Role: FOLLOWER
I20260619 14:57:17.843852 20087 consensus_queue.cc:260] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:17.844487 20087 ts_tablet_manager.cc:1428] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 14:57:17.844846 20085 tablet_bootstrap.cc:492] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: No bootstrap required, opened a new log
I20260619 14:57:17.845069 20085 ts_tablet_manager.cc:1397] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.005s
I20260619 14:57:17.845925 20085 raft_consensus.cc:348] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:17.846169 20085 raft_consensus.cc:374] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:17.846262 20085 raft_consensus.cc:729] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e688dc32f58f4c65829fe056b3a17f2d, State: Initialized, Role: FOLLOWER
I20260619 14:57:17.846535 20085 consensus_queue.cc:260] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:17.847177 20085 ts_tablet_manager.cc:1428] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 14:57:17.849082 20110 tablet_replica.cc:330] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: stopping tablet replica
I20260619 14:57:17.849579 20110 raft_consensus.cc:2229] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:17.850119 20110 raft_consensus.cc:2258] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:17.851203 20109 tablet_replica.cc:330] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: stopping tablet replica
I20260619 14:57:17.851775 20109 raft_consensus.cc:2229] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:17.852422 20109 raft_consensus.cc:2258] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:17.854215 20109 ts_tablet_manager.cc:1905] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:17.855832 20108 ts_tablet_manager.cc:1918] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:17.856083 20108 log.cc:1196] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/687c0a54a710471db9fb12ac9244079b
I20260619 14:57:17.856640 20108 ts_tablet_manager.cc:1939] T 687c0a54a710471db9fb12ac9244079b P b53d293281434901ad4bbe5b453b8942: Deleting consensus metadata
I20260619 14:57:17.858577 19559 catalog_manager.cc:4841] TS b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983): tablet 687c0a54a710471db9fb12ac9244079b (table test-table [id=15c6528a2b3143d4a6461c40e5aac431]) successfully deleted
I20260619 14:57:17.859797 20109 ts_tablet_manager.cc:1918] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:17.859997 20109 log.cc:1196] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/687c0a54a710471db9fb12ac9244079b
I20260619 14:57:17.860415 20109 ts_tablet_manager.cc:1939] T 687c0a54a710471db9fb12ac9244079b P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting consensus metadata
I20260619 14:57:17.861971 19558 catalog_manager.cc:4841] TS cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527): tablet 687c0a54a710471db9fb12ac9244079b (table test-table [id=15c6528a2b3143d4a6461c40e5aac431]) successfully deleted
I20260619 14:57:17.863219 20092 raft_consensus.cc:482] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:57:17.863399 20092 raft_consensus.cc:504] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:17.864723 20110 ts_tablet_manager.cc:1905] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:17.864987 20092 leader_election.cc:288] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063), b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983)
I20260619 14:57:17.872341 20110 ts_tablet_manager.cc:1918] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:17.872537 20110 log.cc:1196] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/687c0a54a710471db9fb12ac9244079b
I20260619 14:57:17.873216 20110 ts_tablet_manager.cc:1939] T 687c0a54a710471db9fb12ac9244079b P e688dc32f58f4c65829fe056b3a17f2d: Deleting consensus metadata
I20260619 14:57:17.875404 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.865889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:36847 (local address 127.11.175.193:34063)
0619 14:57:17.866054 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.866059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.866077 (+ 18us) server_negotiation.cc:407] Connection header received
0619 14:57:17.866153 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.866158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.866234 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.866336 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:17.867126 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.868623 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.871516 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.873321 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.874748 (+ 1427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.874872 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.874958 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.875121 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 14:57:17.875331 19560 catalog_manager.cc:4841] TS e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063): tablet 687c0a54a710471db9fb12ac9244079b (table test-table [id=15c6528a2b3143d4a6461c40e5aac431]) successfully deleted
I20260619 14:57:17.875916 19949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.865282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:34063 (local address 127.11.175.195:36847)
0619 14:57:17.865567 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:17.865587 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:17.865717 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:17.866560 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:17.866567 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:17.866605 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:17.866907 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.866934 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.868848 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.868858 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.871304 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.871323 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.873793 (+ 2470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.873803 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.874132 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.874194 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 14:57:17.874326 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 14:57:17.874543 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260619 14:57:17.876667 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619c1c5ab1404500ab0d463cd8e9a930" candidate_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e688dc32f58f4c65829fe056b3a17f2d" is_pre_election: true
I20260619 14:57:17.876943 19730 raft_consensus.cc:2454] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae97e7c1e0a45708ebb4e4bd08f6254 in term 0.
I20260619 14:57:17.877710 19943 leader_election.cc:302] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cae97e7c1e0a45708ebb4e4bd08f6254, e688dc32f58f4c65829fe056b3a17f2d; no voters:
I20260619 14:57:17.878285 20092 raft_consensus.cc:2781] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:57:17.878429 20092 raft_consensus.cc:482] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:57:17.878530 20092 raft_consensus.cc:3037] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:17.882115 20092 raft_consensus.cc:504] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:17.882997 20092 leader_election.cc:288] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [CANDIDATE]: Term 1 election: Requested vote from peers e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063), b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983)
I20260619 14:57:17.883539 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619c1c5ab1404500ab0d463cd8e9a930" candidate_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e688dc32f58f4c65829fe056b3a17f2d"
I20260619 14:57:17.883801 19730 raft_consensus.cc:3037] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:17.884145 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.866085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:39253 (local address 127.11.175.194:43983)
0619 14:57:17.866252 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.866258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.876845 (+ 10587us) server_negotiation.cc:407] Connection header received
0619 14:57:17.877108 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.877114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.877193 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.877295 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:17.879018 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.880157 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.881941 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.883084 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.883241 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.883322 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.883677 (+ 355us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.883822 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 14:57:17.884907 19949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.865495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:43983 (local address 127.11.175.195:39253)
0619 14:57:17.876734 (+ 11239us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:17.876755 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:17.876882 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:17.878274 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:17.878281 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:17.878324 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:17.878743 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.878761 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.880383 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.880386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.881776 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.881788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.883240 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.883246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.883472 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.883509 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 14:57:17.884397 (+ 888us) client_negotiation.cc:240] Negotiation successful
0619 14:57:17.884561 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11116,"thread_start_us":14140,"threads_started":1}
I20260619 14:57:17.887151 19730 raft_consensus.cc:2454] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae97e7c1e0a45708ebb4e4bd08f6254 in term 1.
I20260619 14:57:17.887647 19869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619c1c5ab1404500ab0d463cd8e9a930" candidate_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b53d293281434901ad4bbe5b453b8942"
I20260619 14:57:17.887620 19870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619c1c5ab1404500ab0d463cd8e9a930" candidate_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b53d293281434901ad4bbe5b453b8942" is_pre_election: true
I20260619 14:57:17.887969 19943 leader_election.cc:302] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cae97e7c1e0a45708ebb4e4bd08f6254, e688dc32f58f4c65829fe056b3a17f2d; no voters:
I20260619 14:57:17.888176 19870 raft_consensus.cc:2454] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae97e7c1e0a45708ebb4e4bd08f6254 in term 0.
I20260619 14:57:17.888278 20092 raft_consensus.cc:2781] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:17.889120 20092 raft_consensus.cc:686] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 LEADER]: Becoming Leader. State: Replica: cae97e7c1e0a45708ebb4e4bd08f6254, State: Running, Role: LEADER
I20260619 14:57:17.889552 20092 consensus_queue.cc:237] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:17.893525 19573 catalog_manager.cc:5495] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 reported cstate change: term changed from 0 to 1, leader changed from <none> to cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195). New cstate: current_term: 1 leader_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } health_report { overall_health: HEALTHY } } }
W20260619 14:57:17.911814 19781 tablet.cc:2369] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 14:57:17 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.254@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:17.962917 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.945337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41139 (local address 127.0.0.1:43404)
0619 14:57:17.946136 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:17.946158 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:17.946317 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:17.946694 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:17.946699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:17.947679 (+ 980us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:17.948016 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.948029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.949046 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.949052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.950608 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:17.950625 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.951435 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.951442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:17.951626 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.952384 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:17.952410 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:17.955753 (+ 3343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:17.959928 (+ 4175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:17.959942 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:17.959951 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:17.960514 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:17.960934 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:17.960939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:17.960943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:17.961110 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:17.961739 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:17.961747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:17.961934 (+ 187us) client_negotiation.cc:769] Sending connection context
0619 14:57:17.962092 (+ 158us) client_negotiation.cc:240] Negotiation successful
0619 14:57:17.962401 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":428,"threads_started":1}
I20260619 14:57:17.963382 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:17.945345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.11.175.254:41139)
0619 14:57:17.945581 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:17.945589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:17.946271 (+ 682us) server_negotiation.cc:407] Connection header received
0619 14:57:17.946438 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:17.946441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:17.946504 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:17.946603 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:17.948183 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.948901 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.950784 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:17.951241 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:17.951337 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:17.956003 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:17.956012 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:17.956019 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:17.956062 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:17.959765 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:17.960667 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:17.960674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:17.960678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:17.960817 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:17.961232 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:17.961238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:17.961242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:17.961601 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:17.961747 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:17.962230 (+ 483us) server_negotiation.cc:299] Negotiation successful
0619 14:57:17.962582 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":84}
I20260619 14:57:17.969619 19573 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:43404:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 14:57:17.971036 19573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:57:17.979579 19990 tablet_service.cc:1467] Processing CreateTablet for tablet aeb025d349524a61b7b0b273412d7010 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa86e6af6fd2437db63a63d7a2fb528c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 14:57:17.980167 19990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb025d349524a61b7b0b273412d7010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:17.982218 19710 tablet_service.cc:1467] Processing CreateTablet for tablet aeb025d349524a61b7b0b273412d7010 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa86e6af6fd2437db63a63d7a2fb528c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 14:57:17.982532 19850 tablet_service.cc:1467] Processing CreateTablet for tablet aeb025d349524a61b7b0b273412d7010 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa86e6af6fd2437db63a63d7a2fb528c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 14:57:17.982833 19710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb025d349524a61b7b0b273412d7010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:17.983332 19850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb025d349524a61b7b0b273412d7010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:17.990561 20087 tablet_bootstrap.cc:492] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: Bootstrap starting.
I20260619 14:57:17.990561 20085 tablet_bootstrap.cc:492] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: Bootstrap starting.
I20260619 14:57:17.991556 20084 tablet_bootstrap.cc:492] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: Bootstrap starting.
I20260619 14:57:17.992447 20087 tablet_bootstrap.cc:654] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:17.992453 20085 tablet_bootstrap.cc:654] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:17.993338 20084 tablet_bootstrap.cc:654] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:18.003595 20087 tablet_bootstrap.cc:492] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: No bootstrap required, opened a new log
I20260619 14:57:18.003805 20087 ts_tablet_manager.cc:1397] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260619 14:57:18.003854 20084 tablet_bootstrap.cc:492] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: No bootstrap required, opened a new log
I20260619 14:57:18.004033 20084 ts_tablet_manager.cc:1397] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260619 14:57:18.004618 20085 tablet_bootstrap.cc:492] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: No bootstrap required, opened a new log
I20260619 14:57:18.004858 20085 ts_tablet_manager.cc:1397] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.001s
I20260619 14:57:18.004698 20087 raft_consensus.cc:348] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.004957 20087 raft_consensus.cc:374] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:18.004858 20084 raft_consensus.cc:348] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.005039 20087 raft_consensus.cc:729] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae97e7c1e0a45708ebb4e4bd08f6254, State: Initialized, Role: FOLLOWER
I20260619 14:57:18.005133 20084 raft_consensus.cc:374] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:18.005231 20084 raft_consensus.cc:729] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b53d293281434901ad4bbe5b453b8942, State: Initialized, Role: FOLLOWER
I20260619 14:57:18.005347 20087 consensus_queue.cc:260] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.005527 20084 consensus_queue.cc:260] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.005820 20092 tablet_replica.cc:439] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } }
I20260619 14:57:18.006122 20092 tablet_replica.cc:442] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:57:18.005985 20103 tablet_replica.cc:439] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } }
I20260619 14:57:18.006354 20103 tablet_replica.cc:442] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:57:18.007189 20087 ts_tablet_manager.cc:1428] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 14:57:18.008230 20084 ts_tablet_manager.cc:1428] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260619 14:57:18.008203 20085 raft_consensus.cc:348] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.008572 20085 raft_consensus.cc:374] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:18.008666 20085 raft_consensus.cc:729] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e688dc32f58f4c65829fe056b3a17f2d, State: Initialized, Role: FOLLOWER
I20260619 14:57:18.008929 20085 consensus_queue.cc:260] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.009599 20085 ts_tablet_manager.cc:1428] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260619 14:57:18.009461 20091 tablet_replica.cc:439] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } }
I20260619 14:57:18.009852 20091 tablet_replica.cc:442] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:57:18.014412 20092 raft_consensus.cc:482] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:57:18.014645 20092 raft_consensus.cc:504] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.015347 20092 leader_election.cc:288] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063), b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983)
I20260619 14:57:18.015910 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb025d349524a61b7b0b273412d7010" candidate_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e688dc32f58f4c65829fe056b3a17f2d" is_pre_election: true
I20260619 14:57:18.015931 19869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb025d349524a61b7b0b273412d7010" candidate_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b53d293281434901ad4bbe5b453b8942" is_pre_election: true
I20260619 14:57:18.016191 19730 raft_consensus.cc:2454] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae97e7c1e0a45708ebb4e4bd08f6254 in term 0.
I20260619 14:57:18.016233 19869 raft_consensus.cc:2454] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae97e7c1e0a45708ebb4e4bd08f6254 in term 0.
I20260619 14:57:18.016774 19942 leader_election.cc:302] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b53d293281434901ad4bbe5b453b8942, cae97e7c1e0a45708ebb4e4bd08f6254, e688dc32f58f4c65829fe056b3a17f2d; no voters:
I20260619 14:57:18.017071 20092 raft_consensus.cc:2781] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:57:18.017181 20092 raft_consensus.cc:482] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:57:18.017305 20092 raft_consensus.cc:3037] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:18.019697 20092 raft_consensus.cc:504] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.020315 20092 leader_election.cc:288] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [CANDIDATE]: Term 1 election: Requested vote from peers e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063), b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983)
I20260619 14:57:18.020781 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb025d349524a61b7b0b273412d7010" candidate_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e688dc32f58f4c65829fe056b3a17f2d"
I20260619 14:57:18.020892 19869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb025d349524a61b7b0b273412d7010" candidate_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b53d293281434901ad4bbe5b453b8942"
I20260619 14:57:18.021046 19730 raft_consensus.cc:3037] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:18.021152 19869 raft_consensus.cc:3037] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:18.023470 19730 raft_consensus.cc:2454] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae97e7c1e0a45708ebb4e4bd08f6254 in term 1.
I20260619 14:57:18.024035 19943 leader_election.cc:302] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cae97e7c1e0a45708ebb4e4bd08f6254, e688dc32f58f4c65829fe056b3a17f2d; no voters:
I20260619 14:57:18.024327 20092 raft_consensus.cc:2781] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:18.024539 20092 raft_consensus.cc:686] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 LEADER]: Becoming Leader. State: Replica: cae97e7c1e0a45708ebb4e4bd08f6254, State: Running, Role: LEADER
I20260619 14:57:18.024616 19869 raft_consensus.cc:2454] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae97e7c1e0a45708ebb4e4bd08f6254 in term 1.
I20260619 14:57:18.024797 20092 consensus_queue.cc:237] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.027212 20115 tablet_replica.cc:439] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: TxnStatusTablet state changed. Reason: New leader cae97e7c1e0a45708ebb4e4bd08f6254. Latest consensus state: current_term: 1 leader_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } }
I20260619 14:57:18.027642 20115 tablet_replica.cc:442] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: This TxnStatusTablet replica's current role is: LEADER
I20260619 14:57:18.028395 20127 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 14:57:18.028705 20127 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 14:57:18.029148 19573 catalog_manager.cc:5495] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 reported cstate change: term changed from 0 to 1, leader changed from <none> to cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195). New cstate: current_term: 1 leader_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } health_report { overall_health: HEALTHY } } }
I20260619 14:57:18.051358 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.044150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49018 (local address 127.11.175.195:38527)
0619 14:57:18.044331 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:18.044336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:18.044357 (+ 21us) server_negotiation.cc:407] Connection header received
0619 14:57:18.044650 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:18.044656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:18.044763 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:18.044929 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:57:18.045343 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.046678 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.048484 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.049156 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.049304 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.049871 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:57:18.050608 (+ 737us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:57:18.050803 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:18.050929 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 14:57:18.051029 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 14:57:18.051363 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.044020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38527 (local address 127.0.0.1:49018)
0619 14:57:18.044231 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:18.044252 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:18.044389 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:18.044978 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:18.044983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:18.044997 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:57:18.045189 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:18.045199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.046823 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.046828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:18.048313 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:18.048330 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.049359 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.049368 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:18.049587 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.049661 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:57:18.050764 (+ 1103us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:57:18.050791 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 14:57:18.050882 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 14:57:18.051042 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 14:57:18.056636 19730 raft_consensus.cc:1261] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Refusing update from remote peer cae97e7c1e0a45708ebb4e4bd08f6254: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:18.056799 19869 raft_consensus.cc:1261] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Refusing update from remote peer cae97e7c1e0a45708ebb4e4bd08f6254: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:18.057621 20092 consensus_queue.cc:1035] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:18.057991 20115 consensus_queue.cc:1035] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:18.067606 20091 tablet_replica.cc:439] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: TxnStatusTablet state changed. Reason: New leader cae97e7c1e0a45708ebb4e4bd08f6254. Latest consensus state: current_term: 1 leader_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } }
I20260619 14:57:18.067960 20091 tablet_replica.cc:442] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:57:18.072588 20092 tablet_replica.cc:439] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } }
I20260619 14:57:18.072957 20092 tablet_replica.cc:442] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: This TxnStatusTablet replica's current role is: LEADER
I20260619 14:57:18.075433 20100 mvcc.cc:205] Tried to move back new op lower bound from 7298584731871064064 to 7298584731753570304. Current Snapshot: MvccSnapshot[applied={T|T < 7298584731871064064}]
I20260619 14:57:18.077232 20103 tablet_replica.cc:439] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: TxnStatusTablet state changed. Reason: New leader cae97e7c1e0a45708ebb4e4bd08f6254. Latest consensus state: current_term: 1 leader_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } }
I20260619 14:57:18.077557 20103 tablet_replica.cc:442] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:57:18.079847 20091 tablet_replica.cc:439] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } }
I20260619 14:57:18.080080 20091 tablet_replica.cc:442] T aeb025d349524a61b7b0b273412d7010 P e688dc32f58f4c65829fe056b3a17f2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:57:18.082902 20115 tablet_replica.cc:439] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } }
I20260619 14:57:18.083164 20115 tablet_replica.cc:442] T aeb025d349524a61b7b0b273412d7010 P cae97e7c1e0a45708ebb4e4bd08f6254: This TxnStatusTablet replica's current role is: LEADER
I20260619 14:57:18.084909 20103 tablet_replica.cc:439] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } } }
I20260619 14:57:18.085171 20103 tablet_replica.cc:442] T aeb025d349524a61b7b0b273412d7010 P b53d293281434901ad4bbe5b453b8942: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 14:57:18.090374 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.080423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49020 (local address 127.11.175.195:38527)
0619 14:57:18.080607 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:18.080613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:18.080632 (+ 19us) server_negotiation.cc:407] Connection header received
0619 14:57:18.080716 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:18.080721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:18.080808 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:18.080897 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:57:18.081480 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.082622 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.087160 (+ 4538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.087901 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.088039 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.089126 (+ 1087us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:57:18.089264 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:57:18.089365 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:18.089986 (+ 621us) server_negotiation.cc:299] Negotiation successful
0619 14:57:18.090068 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 14:57:18.111637 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.103551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:35951 (local address 127.11.175.195:38527)
0619 14:57:18.103760 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:18.103766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:18.104382 (+ 616us) server_negotiation.cc:407] Connection header received
0619 14:57:18.104746 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:18.104754 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:18.104839 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:18.104937 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:57:18.105631 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.106900 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.109125 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.109887 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.110021 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.110709 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:57:18.110876 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:57:18.110973 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:18.111155 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 14:57:18.111248 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":73}
I20260619 14:57:18.111747 20134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.103452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38527 (local address 127.11.175.195:35951)
0619 14:57:18.104249 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:18.104269 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:18.104575 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:18.104972 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:18.104977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:18.104990 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:57:18.105449 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:18.105461 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.107019 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.107027 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:18.108952 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:18.108967 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.110158 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.110164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:18.110370 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.110454 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:57:18.110998 (+ 544us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:57:18.111010 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 14:57:18.111111 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 14:57:18.111269 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":28,"thread_start_us":440,"threads_started":1}
I20260619 14:57:18.124867 19730 raft_consensus.cc:1261] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Refusing update from remote peer cae97e7c1e0a45708ebb4e4bd08f6254: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:18.125622 20115 consensus_queue.cc:1035] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:18.137873 19869 raft_consensus.cc:3037] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:18.141490 19869 raft_consensus.cc:1261] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Refusing update from remote peer cae97e7c1e0a45708ebb4e4bd08f6254: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:18.143831 20092 consensus_queue.cc:1035] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:18.225517 19573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:18.225795 19573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:18.229698 19573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef: Sending DeleteTablet for 3 replicas of tablet 619c1c5ab1404500ab0d463cd8e9a930
I20260619 14:57:18.231000 19850 tablet_service.cc:1514] Processing DeleteTablet for tablet 619c1c5ab1404500ab0d463cd8e9a930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:18 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:53872
I20260619 14:57:18.231367 20108 tablet_replica.cc:330] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942: stopping tablet replica
I20260619 14:57:18.231575 20108 raft_consensus.cc:2229] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:18.231810 20108 raft_consensus.cc:2258] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:18.232040 19710 tablet_service.cc:1514] Processing DeleteTablet for tablet 619c1c5ab1404500ab0d463cd8e9a930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:18 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:60102
I20260619 14:57:18.232364 20110 tablet_replica.cc:330] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: stopping tablet replica
I20260619 14:57:18.232599 20110 raft_consensus.cc:2229] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:18.232792 20110 raft_consensus.cc:2258] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:18.234246 20110 ts_tablet_manager.cc:1905] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:18.234597 20108 ts_tablet_manager.cc:1905] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:18.239001 19990 tablet_service.cc:1514] Processing DeleteTablet for tablet 619c1c5ab1404500ab0d463cd8e9a930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:18 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:49010
I20260619 14:57:18.239426 20109 tablet_replica.cc:330] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254: stopping tablet replica
I20260619 14:57:18.239650 20109 raft_consensus.cc:2229] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:57:18.240213 20109 raft_consensus.cc:2258] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:18.241605 20109 ts_tablet_manager.cc:1905] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:18.242726 20110 ts_tablet_manager.cc:1918] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:57:18.242882 20110 log.cc:1196] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/619c1c5ab1404500ab0d463cd8e9a930
I20260619 14:57:18.243310 20110 ts_tablet_manager.cc:1939] T 619c1c5ab1404500ab0d463cd8e9a930 P e688dc32f58f4c65829fe056b3a17f2d: Deleting consensus metadata
I20260619 14:57:18.244446 19560 catalog_manager.cc:4841] TS e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063): tablet 619c1c5ab1404500ab0d463cd8e9a930 (table test-table [id=47a3cca872484bbdbab0150e5fc0383f]) successfully deleted
I20260619 14:57:18.253831 20108 ts_tablet_manager.cc:1918] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:57:18.254717 20109 ts_tablet_manager.cc:1918] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:57:18.255043 20108 log.cc:1196] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/619c1c5ab1404500ab0d463cd8e9a930
I20260619 14:57:18.255699 20108 ts_tablet_manager.cc:1939] T 619c1c5ab1404500ab0d463cd8e9a930 P b53d293281434901ad4bbe5b453b8942: Deleting consensus metadata
I20260619 14:57:18.255916 20109 log.cc:1196] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/619c1c5ab1404500ab0d463cd8e9a930
I20260619 14:57:18.256466 20109 ts_tablet_manager.cc:1939] T 619c1c5ab1404500ab0d463cd8e9a930 P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting consensus metadata
I20260619 14:57:18.257931 19559 catalog_manager.cc:4841] TS b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983): tablet 619c1c5ab1404500ab0d463cd8e9a930 (table test-table [id=47a3cca872484bbdbab0150e5fc0383f]) successfully deleted
I20260619 14:57:18.259493 19558 catalog_manager.cc:4841] TS cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527): tablet 619c1c5ab1404500ab0d463cd8e9a930 (table test-table [id=47a3cca872484bbdbab0150e5fc0383f]) successfully deleted
Jun 19 14:57:18 dist-test-slave-76rk krb5kdc[19528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 14:57:18 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:18 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 14:57:18.319193 19948 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:18.304744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.11.175.254:41139)
0619 14:57:18.304944 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:18.304951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:18.305580 (+ 629us) server_negotiation.cc:407] Connection header received
0619 14:57:18.305765 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:18.305772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:18.305879 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:18.306009 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:18.308629 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.309772 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.311727 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.312408 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.312559 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.318906 (+ 6347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43410: BlockingRecv error: recv got EOF from 127.0.0.1:43410 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 19 14:57:18 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM
I20260619 14:57:18.341539 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.322541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43422 (local address 127.11.175.254:41139)
0619 14:57:18.322782 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:18.322788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:18.323287 (+ 499us) server_negotiation.cc:407] Connection header received
0619 14:57:18.323519 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:18.323525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:18.323615 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:18.323725 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:18.325359 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.326584 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.328606 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.329311 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.329462 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.334581 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:18.334588 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:18.334593 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:18.334668 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:18.338259 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:18.339222 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:18.339228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:18.339234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:18.339397 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:18.339859 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:18.339865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:18.339871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:18.340265 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:18.340538 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:18.340908 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 14:57:18.341148 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 14:57:18.346012 19573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:57:18.346778 19573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:57:18.354895 19990 tablet_service.cc:1467] Processing CreateTablet for tablet 4a14cd8a0553465884d9e9e097f6e993 (DEFAULT_TABLE table=test-table [id=64866b5fe1d74ab689ebdcb23af89c59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:18.355379 19990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a14cd8a0553465884d9e9e097f6e993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:18.355347 19710 tablet_service.cc:1467] Processing CreateTablet for tablet 4a14cd8a0553465884d9e9e097f6e993 (DEFAULT_TABLE table=test-table [id=64866b5fe1d74ab689ebdcb23af89c59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:18.355847 19710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a14cd8a0553465884d9e9e097f6e993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:18.362701 19850 tablet_service.cc:1467] Processing CreateTablet for tablet 4a14cd8a0553465884d9e9e097f6e993 (DEFAULT_TABLE table=test-table [id=64866b5fe1d74ab689ebdcb23af89c59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:18.363075 20087 tablet_bootstrap.cc:492] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254: Bootstrap starting.
I20260619 14:57:18.363142 19850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a14cd8a0553465884d9e9e097f6e993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:18.364925 20087 tablet_bootstrap.cc:654] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:18.370841 20085 tablet_bootstrap.cc:492] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d: Bootstrap starting.
I20260619 14:57:18.372555 20085 tablet_bootstrap.cc:654] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:18.376649 20084 tablet_bootstrap.cc:492] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942: Bootstrap starting.
I20260619 14:57:18.378491 20084 tablet_bootstrap.cc:654] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:18.380834 20087 tablet_bootstrap.cc:492] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254: No bootstrap required, opened a new log
I20260619 14:57:18.381024 20087 ts_tablet_manager.cc:1397] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.009s
I20260619 14:57:18.381887 20087 raft_consensus.cc:348] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.382197 20087 raft_consensus.cc:374] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:18.382309 20087 raft_consensus.cc:729] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae97e7c1e0a45708ebb4e4bd08f6254, State: Initialized, Role: FOLLOWER
I20260619 14:57:18.383908 20087 consensus_queue.cc:260] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.384601 20087 ts_tablet_manager.cc:1428] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 14:57:18.396358 20084 tablet_bootstrap.cc:492] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942: No bootstrap required, opened a new log
I20260619 14:57:18.396634 20084 ts_tablet_manager.cc:1397] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.005s
I20260619 14:57:18.397482 20084 raft_consensus.cc:348] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.397830 20084 raft_consensus.cc:374] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:18.397948 20085 tablet_bootstrap.cc:492] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d: No bootstrap required, opened a new log
I20260619 14:57:18.397966 20084 raft_consensus.cc:729] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b53d293281434901ad4bbe5b453b8942, State: Initialized, Role: FOLLOWER
I20260619 14:57:18.398128 20085 ts_tablet_manager.cc:1397] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.006s
I20260619 14:57:18.398362 20084 consensus_queue.cc:260] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.399021 20085 raft_consensus.cc:348] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.399324 20085 raft_consensus.cc:374] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:18.399420 20085 raft_consensus.cc:729] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e688dc32f58f4c65829fe056b3a17f2d, State: Initialized, Role: FOLLOWER
I20260619 14:57:18.399649 20085 consensus_queue.cc:260] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.400240 20085 ts_tablet_manager.cc:1428] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 14:57:18.401515 20084 ts_tablet_manager.cc:1428] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260619 14:57:18.422246 20091 raft_consensus.cc:482] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:57:18.422520 20091 raft_consensus.cc:504] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.424367 20091 leader_election.cc:288] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983), cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527)
I20260619 14:57:18.443039 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.425124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:54881 (local address 127.11.175.194:43983)
0619 14:57:18.435164 (+ 10040us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:18.435170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:18.435189 (+ 19us) server_negotiation.cc:407] Connection header received
0619 14:57:18.435265 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:18.435271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:18.435342 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:18.435437 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:18.436409 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.437547 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.440620 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.441857 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.442064 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.442155 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:18.442508 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 14:57:18.442682 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9859,"thread_start_us":9702,"threads_started":1}
I20260619 14:57:18.443037 20154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.424658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:43983 (local address 127.11.175.193:54881)
0619 14:57:18.433424 (+ 8766us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:18.433442 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:18.433596 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:18.435668 (+ 2072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:18.435673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:18.435699 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:18.436094 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:18.436109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.437717 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.437728 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:18.439928 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:18.439943 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.442099 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.442106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:18.442302 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.442342 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 14:57:18.442455 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 14:57:18.442622 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8625,"thread_start_us":15526,"threads_started":1}
I20260619 14:57:18.444352 19870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a14cd8a0553465884d9e9e097f6e993" candidate_uuid: "e688dc32f58f4c65829fe056b3a17f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b53d293281434901ad4bbe5b453b8942" is_pre_election: true
I20260619 14:57:18.444612 19870 raft_consensus.cc:2454] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e688dc32f58f4c65829fe056b3a17f2d in term 0.
I20260619 14:57:18.445173 19662 leader_election.cc:302] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b53d293281434901ad4bbe5b453b8942, e688dc32f58f4c65829fe056b3a17f2d; no voters:
I20260619 14:57:18.445621 20091 raft_consensus.cc:2781] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:57:18.445741 20091 raft_consensus.cc:482] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:57:18.445822 20091 raft_consensus.cc:3037] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:18.448067 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.424826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:53863 (local address 127.11.175.195:38527)
0619 14:57:18.425092 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:18.425102 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:18.441040 (+ 15938us) server_negotiation.cc:407] Connection header received
0619 14:57:18.441205 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:18.441211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:18.441297 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:18.441408 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:18.442047 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.443186 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.445430 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.446814 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.446968 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.447058 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:18.447680 (+ 622us) server_negotiation.cc:299] Negotiation successful
0619 14:57:18.447829 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 14:57:18.448186 20155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.424679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38527 (local address 127.11.175.193:53863)
0619 14:57:18.440888 (+ 16209us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:18.440909 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:18.441053 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:18.441451 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:18.441456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:18.441491 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:18.441843 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:18.441853 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.443340 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.443345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:18.445238 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:18.445253 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.447271 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.447278 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:18.447467 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.447500 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 14:57:18.447604 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 14:57:18.447762 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16076,"thread_start_us":8449,"threads_started":1}
I20260619 14:57:18.449030 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a14cd8a0553465884d9e9e097f6e993" candidate_uuid: "e688dc32f58f4c65829fe056b3a17f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" is_pre_election: true
I20260619 14:57:18.449087 20091 raft_consensus.cc:504] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.449314 20010 raft_consensus.cc:2454] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e688dc32f58f4c65829fe056b3a17f2d in term 0.
I20260619 14:57:18.449915 20091 leader_election.cc:288] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [CANDIDATE]: Term 1 election: Requested vote from peers b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983), cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527)
I20260619 14:57:18.450459 19870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a14cd8a0553465884d9e9e097f6e993" candidate_uuid: "e688dc32f58f4c65829fe056b3a17f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b53d293281434901ad4bbe5b453b8942"
I20260619 14:57:18.450532 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a14cd8a0553465884d9e9e097f6e993" candidate_uuid: "e688dc32f58f4c65829fe056b3a17f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254"
I20260619 14:57:18.450743 19870 raft_consensus.cc:3037] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:18.450770 20010 raft_consensus.cc:3037] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:18.452963 20010 raft_consensus.cc:2454] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e688dc32f58f4c65829fe056b3a17f2d in term 1.
I20260619 14:57:18.453460 19661 leader_election.cc:302] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cae97e7c1e0a45708ebb4e4bd08f6254, e688dc32f58f4c65829fe056b3a17f2d; no voters:
I20260619 14:57:18.453815 20091 raft_consensus.cc:2781] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:18.453876 19870 raft_consensus.cc:2454] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e688dc32f58f4c65829fe056b3a17f2d in term 1.
I20260619 14:57:18.454339 20091 raft_consensus.cc:686] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 1 LEADER]: Becoming Leader. State: Replica: e688dc32f58f4c65829fe056b3a17f2d, State: Running, Role: LEADER
I20260619 14:57:18.454815 20091 consensus_queue.cc:237] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.458792 19571 catalog_manager.cc:5495] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d reported cstate change: term changed from 0 to 1, leader changed from <none> to e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193). New cstate: current_term: 1 leader_uuid: "e688dc32f58f4c65829fe056b3a17f2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260619 14:57:18.479754 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:18.472360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.11.175.193:34063)
0619 14:57:18.473122 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:18.473130 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:18.473162 (+ 32us) server_negotiation.cc:407] Connection header received
0619 14:57:18.473289 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:18.473294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:18.473471 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:18.473684 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:57:18.474464 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.475606 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.477070 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:18.477508 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:18.477658 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:18.478166 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:57:18.478950 (+ 784us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:57:18.479113 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:18.479227 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 14:57:18.479316 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":435,"threads_started":1}
I20260619 14:57:18.483984 20010 raft_consensus.cc:1261] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Refusing update from remote peer e688dc32f58f4c65829fe056b3a17f2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:18.484025 19870 raft_consensus.cc:1261] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Refusing update from remote peer e688dc32f58f4c65829fe056b3a17f2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:18.484789 20091 consensus_queue.cc:1035] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:18.485108 20157 consensus_queue.cc:1035] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:18.505179 19571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:18.505463 19571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:18.509719 19571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef: Sending DeleteTablet for 3 replicas of tablet 4a14cd8a0553465884d9e9e097f6e993
I20260619 14:57:18.510857 19850 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a14cd8a0553465884d9e9e097f6e993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:18 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:53872
I20260619 14:57:18.511224 20108 tablet_replica.cc:330] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942: stopping tablet replica
I20260619 14:57:18.511421 20108 raft_consensus.cc:2229] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:18.511675 20108 raft_consensus.cc:2258] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:18.511732 19710 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a14cd8a0553465884d9e9e097f6e993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:18 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:60102
I20260619 14:57:18.512012 20110 tablet_replica.cc:330] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d: stopping tablet replica
I20260619 14:57:18.512199 20110 raft_consensus.cc:2229] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:57:18.512725 20110 raft_consensus.cc:2258] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:18.513008 20108 ts_tablet_manager.cc:1905] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:18.513645 19990 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a14cd8a0553465884d9e9e097f6e993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:18 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:49010
I20260619 14:57:18.515388 19571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:57:18.516165 19571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:57:18.518129 20110 ts_tablet_manager.cc:1905] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:18.518326 20109 tablet_replica.cc:330] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254: stopping tablet replica
I20260619 14:57:18.518524 20109 raft_consensus.cc:2229] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:18.518816 20109 raft_consensus.cc:2258] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:18.523901 20109 ts_tablet_manager.cc:1905] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:18.524350 19710 tablet_service.cc:1467] Processing CreateTablet for tablet 834307003d79488a8ba8b893836c4279 (DEFAULT_TABLE table=test-table [id=47e9ad9ebeec413d89fd7d30f4dd313c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:18.524351 19850 tablet_service.cc:1467] Processing CreateTablet for tablet 834307003d79488a8ba8b893836c4279 (DEFAULT_TABLE table=test-table [id=47e9ad9ebeec413d89fd7d30f4dd313c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:18.524780 19850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 834307003d79488a8ba8b893836c4279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:18.524780 19710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 834307003d79488a8ba8b893836c4279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:18.528658 19990 tablet_service.cc:1467] Processing CreateTablet for tablet 834307003d79488a8ba8b893836c4279 (DEFAULT_TABLE table=test-table [id=47e9ad9ebeec413d89fd7d30f4dd313c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:18.529165 19990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 834307003d79488a8ba8b893836c4279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:18.532080 20085 tablet_bootstrap.cc:492] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d: Bootstrap starting.
I20260619 14:57:18.532840 20108 ts_tablet_manager.cc:1918] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:18.533030 20084 tablet_bootstrap.cc:492] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942: Bootstrap starting.
I20260619 14:57:18.533178 20108 log.cc:1196] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/4a14cd8a0553465884d9e9e097f6e993
I20260619 14:57:18.533732 20108 ts_tablet_manager.cc:1939] T 4a14cd8a0553465884d9e9e097f6e993 P b53d293281434901ad4bbe5b453b8942: Deleting consensus metadata
I20260619 14:57:18.533946 20085 tablet_bootstrap.cc:654] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:18.534934 20084 tablet_bootstrap.cc:654] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:18.536301 19559 catalog_manager.cc:4841] TS b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983): tablet 4a14cd8a0553465884d9e9e097f6e993 (table test-table [id=64866b5fe1d74ab689ebdcb23af89c59]) successfully deleted
I20260619 14:57:18.547245 20084 tablet_bootstrap.cc:492] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942: No bootstrap required, opened a new log
I20260619 14:57:18.547444 20084 ts_tablet_manager.cc:1397] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.007s
I20260619 14:57:18.548292 20084 raft_consensus.cc:348] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.548585 20084 raft_consensus.cc:374] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:18.548672 20084 raft_consensus.cc:729] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b53d293281434901ad4bbe5b453b8942, State: Initialized, Role: FOLLOWER
I20260619 14:57:18.548931 20084 consensus_queue.cc:260] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.549513 20084 ts_tablet_manager.cc:1428] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260619 14:57:18.561111 20109 ts_tablet_manager.cc:1918] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:18.561276 20109 log.cc:1196] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/4a14cd8a0553465884d9e9e097f6e993
I20260619 14:57:18.561733 20109 ts_tablet_manager.cc:1939] T 4a14cd8a0553465884d9e9e097f6e993 P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting consensus metadata
I20260619 14:57:18.562919 19558 catalog_manager.cc:4841] TS cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527): tablet 4a14cd8a0553465884d9e9e097f6e993 (table test-table [id=64866b5fe1d74ab689ebdcb23af89c59]) successfully deleted
I20260619 14:57:18.566968 20087 tablet_bootstrap.cc:492] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254: Bootstrap starting.
I20260619 14:57:18.568820 20087 tablet_bootstrap.cc:654] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:18.575965 20110 ts_tablet_manager.cc:1918] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:18.576274 20110 log.cc:1196] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/4a14cd8a0553465884d9e9e097f6e993
I20260619 14:57:18.576726 20110 ts_tablet_manager.cc:1939] T 4a14cd8a0553465884d9e9e097f6e993 P e688dc32f58f4c65829fe056b3a17f2d: Deleting consensus metadata
I20260619 14:57:18.577982 19560 catalog_manager.cc:4841] TS e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063): tablet 4a14cd8a0553465884d9e9e097f6e993 (table test-table [id=64866b5fe1d74ab689ebdcb23af89c59]) successfully deleted
I20260619 14:57:18.579813 20085 tablet_bootstrap.cc:492] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d: No bootstrap required, opened a new log
I20260619 14:57:18.580099 20085 ts_tablet_manager.cc:1397] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d: Time spent bootstrapping tablet: real 0.048s user 0.011s sys 0.002s
I20260619 14:57:18.580233 20087 tablet_bootstrap.cc:492] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254: No bootstrap required, opened a new log
I20260619 14:57:18.580420 20087 ts_tablet_manager.cc:1397] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.001s
I20260619 14:57:18.581102 20085 raft_consensus.cc:348] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.581235 20087 raft_consensus.cc:348] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.581425 20085 raft_consensus.cc:374] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:18.581502 20087 raft_consensus.cc:374] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:18.581579 20087 raft_consensus.cc:729] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae97e7c1e0a45708ebb4e4bd08f6254, State: Initialized, Role: FOLLOWER
I20260619 14:57:18.581625 20085 raft_consensus.cc:729] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e688dc32f58f4c65829fe056b3a17f2d, State: Initialized, Role: FOLLOWER
I20260619 14:57:18.581812 20087 consensus_queue.cc:260] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.582306 20087 ts_tablet_manager.cc:1428] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260619 14:57:18.585175 20085 consensus_queue.cc:260] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.585759 20085 ts_tablet_manager.cc:1428] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260619 14:57:18.885172 20157 raft_consensus.cc:482] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:57:18.885489 20157 raft_consensus.cc:504] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.886441 20157 leader_election.cc:288] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983), cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527)
I20260619 14:57:18.887337 19870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834307003d79488a8ba8b893836c4279" candidate_uuid: "e688dc32f58f4c65829fe056b3a17f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b53d293281434901ad4bbe5b453b8942" is_pre_election: true
I20260619 14:57:18.887410 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834307003d79488a8ba8b893836c4279" candidate_uuid: "e688dc32f58f4c65829fe056b3a17f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" is_pre_election: true
I20260619 14:57:18.887636 19870 raft_consensus.cc:2454] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e688dc32f58f4c65829fe056b3a17f2d in term 0.
I20260619 14:57:18.887678 20010 raft_consensus.cc:2454] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e688dc32f58f4c65829fe056b3a17f2d in term 0.
I20260619 14:57:18.888247 19661 leader_election.cc:302] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cae97e7c1e0a45708ebb4e4bd08f6254, e688dc32f58f4c65829fe056b3a17f2d; no voters:
I20260619 14:57:18.888724 20157 raft_consensus.cc:2781] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:57:18.888854 20157 raft_consensus.cc:482] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:57:18.889004 20157 raft_consensus.cc:3037] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:18.892961 20157 raft_consensus.cc:504] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.894371 19870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834307003d79488a8ba8b893836c4279" candidate_uuid: "e688dc32f58f4c65829fe056b3a17f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b53d293281434901ad4bbe5b453b8942"
I20260619 14:57:18.894452 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834307003d79488a8ba8b893836c4279" candidate_uuid: "e688dc32f58f4c65829fe056b3a17f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254"
I20260619 14:57:18.894618 19870 raft_consensus.cc:3037] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:18.894757 20010 raft_consensus.cc:3037] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:18.898020 19870 raft_consensus.cc:2454] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e688dc32f58f4c65829fe056b3a17f2d in term 1.
I20260619 14:57:18.898365 20010 raft_consensus.cc:2454] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e688dc32f58f4c65829fe056b3a17f2d in term 1.
I20260619 14:57:18.898828 19662 leader_election.cc:302] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b53d293281434901ad4bbe5b453b8942, e688dc32f58f4c65829fe056b3a17f2d; no voters:
I20260619 14:57:18.899261 20091 raft_consensus.cc:2781] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:18.899539 20157 leader_election.cc:288] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [CANDIDATE]: Term 1 election: Requested vote from peers b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983), cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527)
I20260619 14:57:18.902017 20091 raft_consensus.cc:686] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 1 LEADER]: Becoming Leader. State: Replica: e688dc32f58f4c65829fe056b3a17f2d, State: Running, Role: LEADER
I20260619 14:57:18.902385 20091 consensus_queue.cc:237] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } }
I20260619 14:57:18.907809 19571 catalog_manager.cc:5495] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d reported cstate change: term changed from 0 to 1, leader changed from <none> to e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193). New cstate: current_term: 1 leader_uuid: "e688dc32f58f4c65829fe056b3a17f2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e688dc32f58f4c65829fe056b3a17f2d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 34063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260619 14:57:19.028731 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:19.021439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38527 (local address 127.11.175.193:52603)
0619 14:57:19.022513 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:19.022530 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:19.022734 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:19.023167 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:19.023170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:19.023181 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:57:19.023390 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:19.023397 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.024947 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.024953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:19.026100 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:19.026123 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.027336 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.027341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:19.027490 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:19.027648 (+ 158us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:57:19.028227 (+ 579us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:57:19.028239 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 14:57:19.028313 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 14:57:19.028432 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"thread_start_us":765,"threads_started":1}
I20260619 14:57:19.028920 20177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:19.021650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:52603 (local address 127.11.175.195:38527)
0619 14:57:19.022243 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:19.022250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:19.022656 (+ 406us) server_negotiation.cc:407] Connection header received
0619 14:57:19.022881 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:19.022887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:19.022980 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:19.023093 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:57:19.023556 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.024748 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.026266 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.027094 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.027234 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:19.027942 (+ 708us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:57:19.028106 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:57:19.028207 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:19.028384 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 14:57:19.028492 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":293,"threads_started":1}
I20260619 14:57:19.038312 20010 raft_consensus.cc:1261] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Refusing update from remote peer e688dc32f58f4c65829fe056b3a17f2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:19.038529 19870 raft_consensus.cc:1261] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Refusing update from remote peer e688dc32f58f4c65829fe056b3a17f2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:19.039057 20167 consensus_queue.cc:1035] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae97e7c1e0a45708ebb4e4bd08f6254" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:19.039458 20091 consensus_queue.cc:1035] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b53d293281434901ad4bbe5b453b8942" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:19.076256 20183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:19.068157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:53703 (local address 127.11.175.193:34063)
0619 14:57:19.069241 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:19.069245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:19.069267 (+ 22us) server_negotiation.cc:407] Connection header received
0619 14:57:19.069331 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:19.069335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:19.069413 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:19.069553 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:57:19.070235 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.071390 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.073686 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.074263 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.074473 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:19.075158 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:57:19.075285 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:57:19.075433 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:19.075631 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 14:57:19.075726 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":960,"thread_start_us":829,"threads_started":1}
I20260619 14:57:19.076246 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:19.067727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:34063 (local address 127.11.175.195:53703)
0619 14:57:19.068612 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:19.068635 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:19.068783 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:19.069583 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:19.069589 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:19.069606 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 14:57:19.069997 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:19.070007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.071609 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.071615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:19.073430 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:19.073450 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.074547 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.074555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:19.074821 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:19.074927 (+ 106us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 14:57:19.075441 (+ 514us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 14:57:19.075460 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 14:57:19.075563 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 14:57:19.075756 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":573,"threads_started":1}
I20260619 14:57:19.112886 19571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:19.113170 19571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:19.116355 19571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 821c6429344e4ec6aad943f1ea5b03ef: Sending DeleteTablet for 3 replicas of tablet 834307003d79488a8ba8b893836c4279
I20260619 14:57:19.117265 19850 tablet_service.cc:1514] Processing DeleteTablet for tablet 834307003d79488a8ba8b893836c4279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:19 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:53872
I20260619 14:57:19.117347 19990 tablet_service.cc:1514] Processing DeleteTablet for tablet 834307003d79488a8ba8b893836c4279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:19 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:49010
I20260619 14:57:19.117467 19710 tablet_service.cc:1514] Processing DeleteTablet for tablet 834307003d79488a8ba8b893836c4279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:19 UTC) from {username='oryx', principal='oryx/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:60102
I20260619 14:57:19.119073 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 19644
I20260619 14:57:19.121469 20185 tablet_replica.cc:330] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942: stopping tablet replica
I20260619 14:57:19.121845 20185 raft_consensus.cc:2229] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:19.122267 20185 raft_consensus.cc:2258] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:19.123778 20185 ts_tablet_manager.cc:1905] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:19.128597 20187 tablet_replica.cc:330] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d: stopping tablet replica
I20260619 14:57:19.128860 20187 raft_consensus.cc:2229] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:57:19.129062 20186 tablet_replica.cc:330] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254: stopping tablet replica
I20260619 14:57:19.129274 20187 raft_consensus.cc:2258] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:19.129274 20186 raft_consensus.cc:2229] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:19.129568 20186 raft_consensus.cc:2258] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:19.130625 20186 ts_tablet_manager.cc:1905] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:19.131294 20187 ts_tablet_manager.cc:1905] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:19.136735 20185 ts_tablet_manager.cc:1918] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:57:19.136904 20185 log.cc:1196] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/834307003d79488a8ba8b893836c4279
I20260619 14:57:19.137074 20186 ts_tablet_manager.cc:1918] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:57:19.137260 20186 log.cc:1196] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/834307003d79488a8ba8b893836c4279
I20260619 14:57:19.137355 20185 ts_tablet_manager.cc:1939] T 834307003d79488a8ba8b893836c4279 P b53d293281434901ad4bbe5b453b8942: Deleting consensus metadata
I20260619 14:57:19.137674 20186 ts_tablet_manager.cc:1939] T 834307003d79488a8ba8b893836c4279 P cae97e7c1e0a45708ebb4e4bd08f6254: Deleting consensus metadata
I20260619 14:57:19.137982 20187 ts_tablet_manager.cc:1918] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 14:57:19.138118 20187 log.cc:1196] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/834307003d79488a8ba8b893836c4279
I20260619 14:57:19.138540 20187 ts_tablet_manager.cc:1939] T 834307003d79488a8ba8b893836c4279 P e688dc32f58f4c65829fe056b3a17f2d: Deleting consensus metadata
I20260619 14:57:19.138537 19559 catalog_manager.cc:4841] TS b53d293281434901ad4bbe5b453b8942 (127.11.175.194:43983): tablet 834307003d79488a8ba8b893836c4279 (table test-table [id=47e9ad9ebeec413d89fd7d30f4dd313c]) successfully deleted
I20260619 14:57:19.138877 19558 catalog_manager.cc:4841] TS cae97e7c1e0a45708ebb4e4bd08f6254 (127.11.175.195:38527): tablet 834307003d79488a8ba8b893836c4279 (table test-table [id=47e9ad9ebeec413d89fd7d30f4dd313c]) successfully deleted
I20260619 14:57:19.139614 19560 catalog_manager.cc:4841] TS e688dc32f58f4c65829fe056b3a17f2d (127.11.175.193:34063): tablet 834307003d79488a8ba8b893836c4279 (table test-table [id=47e9ad9ebeec413d89fd7d30f4dd313c]) successfully deleted
Jun 19 14:57:19 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.193@KRBTEST.COM
I20260619 14:57:19.149740 20183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:19.121061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.11.175.193:34063)
0619 14:57:19.121308 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:19.121319 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:19.128407 (+ 7088us) server_negotiation.cc:407] Connection header received
0619 14:57:19.128611 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:19.128621 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:19.128753 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:19.128886 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:19.130977 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.132107 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.134053 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.134778 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.134910 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:19.143596 (+ 8686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:19.143614 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:19.143630 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:19.143669 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:19.146613 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:19.147465 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:19.147473 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:19.147475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:19.147613 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:19.148110 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:19.148114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:19.148116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:19.148478 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:19.148735 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:19.149130 (+ 395us) server_negotiation.cc:299] Negotiation successful
0619 14:57:19.149330 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 14:57:19.150511 19775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:19.278599 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 19644
I20260619 14:57:19.303067 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 19784
Jun 19 14:57:19 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.194@KRBTEST.COM
I20260619 14:57:19.319332 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:19.303894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53882 (local address 127.11.175.194:43983)
0619 14:57:19.304434 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:19.304439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:19.304454 (+ 15us) server_negotiation.cc:407] Connection header received
0619 14:57:19.304508 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:19.304512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:19.304589 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:19.304698 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:19.306175 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.307174 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.308571 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.308981 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.309083 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:19.313442 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:19.313469 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:19.313486 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:19.313535 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:19.316580 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:19.317343 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:19.317350 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:19.317352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:19.317489 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:19.317843 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:19.317846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:19.317848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:19.318200 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:19.318408 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:19.318778 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 14:57:19.318960 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":310,"threads_started":1}
I20260619 14:57:19.320040 19915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:19.445497 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 19784
I20260619 14:57:19.468815 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 19924
Jun 19 14:57:19 dist-test-slave-76rk krb5kdc[19528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.175.195@KRBTEST.COM
I20260619 14:57:19.489090 20177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:19.469760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49026 (local address 127.11.175.195:38527)
0619 14:57:19.469943 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:19.469948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:19.469967 (+ 19us) server_negotiation.cc:407] Connection header received
0619 14:57:19.470126 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:19.470130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:19.470203 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:19.470292 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:19.471892 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.472996 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.475114 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:19.475561 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:19.475696 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:19.481252 (+ 5556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:19.481270 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:19.481286 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:19.481329 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:19.485718 (+ 4389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:19.486678 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:19.486690 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:19.486693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:19.486879 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:19.487306 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:19.487311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:19.487314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:19.487811 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:19.488150 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:19.488512 (+ 362us) server_negotiation.cc:299] Negotiation successful
0619 14:57:19.488739 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 14:57:19.489756 20055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:19.639662 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:19.637751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:43983 (local address 127.11.175.195:53511)
0619 14:57:19.638787 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:19.639281 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.194:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":898,"threads_started":1}
I20260619 14:57:19.640159 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 19924
I20260619 14:57:19.666281 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 19541
I20260619 14:57:19.667614 19613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:19.787757 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 19541
2026-06-19T14:57:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5881 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:57:19 dist-test-slave-76rk krb5kdc[20198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:57:19 dist-test-slave-76rk krb5kdc[20198](info): set up 2 sockets
Jun 19 14:57:19 dist-test-slave-76rk krb5kdc[20198](info): commencing operation
krb5kdc: starting...
W20260619 14:57:20.923990 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:57:21 dist-test-slave-76rk krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:57:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:57:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:57:21.187356 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38177
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:21.537335 20211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:21.537755 20211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:21.537935 20211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:21.546240 20211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:21.546365 20211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:21.546412 20211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:21.546535 20211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:21.546607 20211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:21.557256 20211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:21.558573 20211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:21.560019 20211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:21.573585 20216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:21.573499 20219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:21.573544 20217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:21.574328 20211 server_base.cc:1034] running on GCE node
I20260619 14:57:21.575306 20211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:21.576958 20211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:21.578240 20211 hybrid_clock.cc:648] HybridClock initialized: now 1781881041578195 us; error 38 us; skew 500 ppm
I20260619 14:57:21.578712 20211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T14:57:21Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1860 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:57:21 dist-test-slave-76rk krb5kdc[20225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:57:21 dist-test-slave-76rk krb5kdc[20225](info): set up 2 sockets
Jun 19 14:57:21 dist-test-slave-76rk krb5kdc[20225](info): commencing operation
krb5kdc: starting...
W20260619 14:57:22.771734 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.008s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:57:22 dist-test-slave-76rk krb5kdc[20225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:57:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:57:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:57:23.019711 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38869
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:23.356942 20238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:23.357349 20238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:23.357527 20238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:23.364946 20238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:23.365142 20238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:23.365262 20238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:23.365370 20238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:23.365437 20238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:23.375571 20238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:23.377010 20238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:23.378387 20238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:23.391026 20246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:23.390993 20244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:23.391304 20243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:23.391949 20238 server_base.cc:1034] running on GCE node
I20260619 14:57:23.392838 20238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:23.394779 20238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:23.396037 20238 hybrid_clock.cc:648] HybridClock initialized: now 1781881043395967 us; error 60 us; skew 500 ppm
I20260619 14:57:23.396436 20238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:23 dist-test-slave-76rk krb5kdc[20225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881043, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:23.399966 20238 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:57:23.401695 20238 webserver.cc:460] Webserver started at http://127.11.175.254:40895/ using document root <none> and password file <none>
I20260619 14:57:23.402449 20238 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:23.402614 20238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:23.402981 20238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:23.406093 20238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "d51711a8ea134400b1fa3af77194b389"
format_stamp: "Formatted at 2026-06-19 14:57:23 on dist-test-slave-76rk"
server_key: "0638988287f0b9583112680c0a4e1480"
server_key_iv: "d74125037bcc7ad2deabc6cb923b1fe9"
server_key_version: "encryptionkey@0"
I20260619 14:57:23.406982 20238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "d51711a8ea134400b1fa3af77194b389"
format_stamp: "Formatted at 2026-06-19 14:57:23 on dist-test-slave-76rk"
server_key: "0638988287f0b9583112680c0a4e1480"
server_key_iv: "d74125037bcc7ad2deabc6cb923b1fe9"
server_key_version: "encryptionkey@0"
I20260619 14:57:23.412581 20238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 14:57:23.416313 20253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:23.417699 20238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 14:57:23.417904 20238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "d51711a8ea134400b1fa3af77194b389"
format_stamp: "Formatted at 2026-06-19 14:57:23 on dist-test-slave-76rk"
server_key: "0638988287f0b9583112680c0a4e1480"
server_key_iv: "d74125037bcc7ad2deabc6cb923b1fe9"
server_key_version: "encryptionkey@0"
I20260619 14:57:23.418165 20238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:23.438611 20238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:23.443367 20238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:23.443677 20238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:23.463618 20238 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:38869
I20260619 14:57:23.463667 20304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:38869 every 8 connection(s)
I20260619 14:57:23.465193 20238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:57:23.469357 20305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:23.473568 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 20238
I20260619 14:57:23.473829 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:57:23.474475 11967 external_mini_cluster.cc:1428] Setting key 2c12b2a8adda93721b38422620643eaa
I20260619 14:57:23.481652 20305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389: Bootstrap starting.
I20260619 14:57:23.485548 20305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:23.486840 20305 log.cc:824] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:23.490324 20305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389: No bootstrap required, opened a new log
Jun 19 14:57:23 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:23.500111 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:23.477045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.11.175.254:38869)
0619 14:57:23.477948 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:23.477960 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:23.478007 (+ 47us) server_negotiation.cc:407] Connection header received
0619 14:57:23.479860 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:23.479885 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:23.480294 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:23.480830 (+ 536us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:23.482840 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:23.484234 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:23.486236 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:23.487098 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:23.487234 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:23.492297 (+ 5063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:23.492305 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:23.492319 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:23.492356 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:23.495703 (+ 3347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:23.496561 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:23.496568 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:23.496578 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:23.496757 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:23.497165 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:23.497171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:23.497173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:23.497795 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:23.498033 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:23.498510 (+ 477us) server_negotiation.cc:299] Negotiation successful
0619 14:57:23.499062 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":412,"threads_started":1}
I20260619 14:57:23.500216 20305 raft_consensus.cc:348] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } }
I20260619 14:57:23.500705 20305 raft_consensus.cc:374] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:23.500851 20305 raft_consensus.cc:729] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51711a8ea134400b1fa3af77194b389, State: Initialized, Role: FOLLOWER
I20260619 14:57:23.501497 20305 consensus_queue.cc:260] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } }
I20260619 14:57:23.501754 20305 raft_consensus.cc:388] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:57:23.501906 20305 raft_consensus.cc:482] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:57:23.502104 20305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:23.504647 20305 raft_consensus.cc:504] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } }
I20260619 14:57:23.505136 20305 leader_election.cc:302] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d51711a8ea134400b1fa3af77194b389; no voters:
I20260619 14:57:23.505722 20305 leader_election.cc:288] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:57:23.505873 20310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:23.506599 20310 raft_consensus.cc:686] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 1 LEADER]: Becoming Leader. State: Replica: d51711a8ea134400b1fa3af77194b389, State: Running, Role: LEADER
I20260619 14:57:23.507176 20310 consensus_queue.cc:237] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } }
I20260619 14:57:23.507949 20305 sys_catalog.cc:564] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:57:23.516314 20312 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d51711a8ea134400b1fa3af77194b389. Latest consensus state: current_term: 1 leader_uuid: "d51711a8ea134400b1fa3af77194b389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } } }
I20260619 14:57:23.516687 20312 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:23.517283 20311 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d51711a8ea134400b1fa3af77194b389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } } }
I20260619 14:57:23.517540 20311 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:23.518126 20320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:57:23.523258 20320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:57:23.532068 20320 catalog_manager.cc:1347] Generated new cluster ID: a38aba547970426da91d0160e831a90b
I20260619 14:57:23.532214 20320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:57:23.562831 20320 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:57:23.564270 20320 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:57:23.573997 20320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389: Generated new TSK 0
I20260619 14:57:23.575130 20320 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:57:23.638770 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38869
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:23.988965 20331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:23.989254 20331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:23.989414 20331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:23.996860 20331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:23.996990 20331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:23.997191 20331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:57:24.007496 20331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:24.008867 20331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:24.010251 20331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:24.023897 20336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:24.024014 20337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:24.024034 20339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:24.025264 20331 server_base.cc:1034] running on GCE node
I20260619 14:57:24.025836 20331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:24.026803 20331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:24.028002 20331 hybrid_clock.cc:648] HybridClock initialized: now 1781881044027938 us; error 56 us; skew 500 ppm
I20260619 14:57:24.028348 20331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:24 dist-test-slave-76rk krb5kdc[20225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:24.031584 20331 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:57:24.033023 20331 webserver.cc:460] Webserver started at http://127.11.175.193:36193/ using document root <none> and password file <none>
I20260619 14:57:24.033694 20331 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:24.033788 20331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:24.034029 20331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:24.037184 20331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "79743877f62d4ed4aa91a2ed31ed5343"
format_stamp: "Formatted at 2026-06-19 14:57:24 on dist-test-slave-76rk"
server_key: "fd9a9f8998b71d4843e06726e434494c"
server_key_iv: "48a78bd751cfaa8b29b8c34811be7c58"
server_key_version: "encryptionkey@0"
I20260619 14:57:24.038084 20331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "79743877f62d4ed4aa91a2ed31ed5343"
format_stamp: "Formatted at 2026-06-19 14:57:24 on dist-test-slave-76rk"
server_key: "fd9a9f8998b71d4843e06726e434494c"
server_key_iv: "48a78bd751cfaa8b29b8c34811be7c58"
server_key_version: "encryptionkey@0"
I20260619 14:57:24.043601 20331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 14:57:24.047483 20346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:24.049015 20331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:57:24.049211 20331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "79743877f62d4ed4aa91a2ed31ed5343"
format_stamp: "Formatted at 2026-06-19 14:57:24 on dist-test-slave-76rk"
server_key: "fd9a9f8998b71d4843e06726e434494c"
server_key_iv: "48a78bd751cfaa8b29b8c34811be7c58"
server_key_version: "encryptionkey@0"
I20260619 14:57:24.049412 20331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:24.067570 20331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:24.073155 20331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:24.073549 20331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:24.074735 20331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:24.076396 20331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:24.076486 20331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:24.076704 20331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:24.076797 20331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:24.112236 20331 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:37811
I20260619 14:57:24.112272 20458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:37811 every 8 connection(s)
I20260619 14:57:24.113513 20331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:57:24.123157 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 20331
I20260619 14:57:24.123382 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:57:24.124131 11967 external_mini_cluster.cc:1428] Setting key d7b0b5a3b29d376269ca4d0cce1e6366
Jun 19 14:57:24 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:24.134289 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:24.116150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:48837 (local address 127.11.175.254:38869)
0619 14:57:24.116616 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:24.116621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:24.117559 (+ 938us) server_negotiation.cc:407] Connection header received
0619 14:57:24.119475 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:24.119481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:24.119577 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:24.119697 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:24.121168 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:24.122011 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:24.123859 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:24.124321 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:24.124455 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:24.128374 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:24.128379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:24.128382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:24.128415 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:24.131146 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:24.131900 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:24.131909 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:24.131912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:24.132032 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:24.132504 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:24.132514 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:24.132516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:24.132792 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:24.132952 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:24.133691 (+ 739us) server_negotiation.cc:299] Negotiation successful
0619 14:57:24.133928 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":250,"threads_started":1}
I20260619 14:57:24.135375 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:24.116714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:48837)
0619 14:57:24.117404 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:24.117444 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:24.119202 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:24.119930 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:24.119938 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:24.120438 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:24.120995 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:24.121012 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:24.122210 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:24.122218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:24.123718 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:24.123731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:24.124569 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:24.124576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:24.124888 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:24.125641 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:24.125670 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:24.128124 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:24.131306 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:24.131315 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:24.131333 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:24.131767 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:24.132171 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:24.132176 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:24.132181 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:24.132381 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:24.132926 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:24.132934 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:24.133299 (+ 365us) client_negotiation.cc:769] Sending connection context
0619 14:57:24.133573 (+ 274us) client_negotiation.cc:240] Negotiation successful
0619 14:57:24.134147 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":268,"threads_started":1}
I20260619 14:57:24.137145 20459 heartbeater.cc:347] Connected to a master server at 127.11.175.254:38869
I20260619 14:57:24.137485 20459 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:24.138239 20459 heartbeater.cc:511] Master 127.11.175.254:38869 requested a full tablet report, sending...
I20260619 14:57:24.140681 20270 ts_manager.cc:194] Registered new tserver with Master: 79743877f62d4ed4aa91a2ed31ed5343 (127.11.175.193:37811)
I20260619 14:57:24.143013 20270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:48837
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:57:24.178611 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38869
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:24.522125 20465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:24.522534 20465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:24.522733 20465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:24.530098 20465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:24.530268 20465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:24.530449 20465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:57:24.540577 20465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:24.541946 20465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:24.543388 20465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:24.557731 20470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:24.558259 20471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:24.560292 20473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:24.560221 20465 server_base.cc:1034] running on GCE node
I20260619 14:57:24.561097 20465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:24.562165 20465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:24.563414 20465 hybrid_clock.cc:648] HybridClock initialized: now 1781881044563354 us; error 45 us; skew 500 ppm
I20260619 14:57:24.563807 20465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:24 dist-test-slave-76rk krb5kdc[20225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:24.573367 20465 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:57:24.575349 20465 webserver.cc:460] Webserver started at http://127.11.175.194:36659/ using document root <none> and password file <none>
I20260619 14:57:24.576153 20465 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:24.576318 20465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:24.576658 20465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:24.579784 20465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "93f4cc31f9e44863a59c256a25f71015"
format_stamp: "Formatted at 2026-06-19 14:57:24 on dist-test-slave-76rk"
server_key: "39e0bcfa4e6fcad6d07f3f8e74dd0875"
server_key_iv: "169e9fc6d4cd5a81263116a7120813e8"
server_key_version: "encryptionkey@0"
I20260619 14:57:24.580586 20465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "93f4cc31f9e44863a59c256a25f71015"
format_stamp: "Formatted at 2026-06-19 14:57:24 on dist-test-slave-76rk"
server_key: "39e0bcfa4e6fcad6d07f3f8e74dd0875"
server_key_iv: "169e9fc6d4cd5a81263116a7120813e8"
server_key_version: "encryptionkey@0"
I20260619 14:57:24.586040 20465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 14:57:24.590106 20480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:24.591612 20465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 14:57:24.591821 20465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "93f4cc31f9e44863a59c256a25f71015"
format_stamp: "Formatted at 2026-06-19 14:57:24 on dist-test-slave-76rk"
server_key: "39e0bcfa4e6fcad6d07f3f8e74dd0875"
server_key_iv: "169e9fc6d4cd5a81263116a7120813e8"
server_key_version: "encryptionkey@0"
I20260619 14:57:24.592083 20465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:24.610105 20465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:24.614913 20465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:24.615304 20465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:24.616485 20465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:24.618026 20465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:24.618149 20465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:24.618332 20465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:24.618424 20465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:24.655397 20465 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:39739
I20260619 14:57:24.655419 20592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:39739 every 8 connection(s)
I20260619 14:57:24.656873 20465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:57:24.663589 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 20465
I20260619 14:57:24.663767 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:57:24.664260 11967 external_mini_cluster.cc:1428] Setting key 13ca96d06445e0fcfa5515a45ef7225f
Jun 19 14:57:24 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:24.678962 20595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:24.659708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:47059 (local address 127.11.175.254:38869)
0619 14:57:24.660223 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:24.660227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:24.661136 (+ 909us) server_negotiation.cc:407] Connection header received
0619 14:57:24.663209 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:24.663216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:24.663328 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:24.663481 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:24.665439 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:24.666330 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:24.668090 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:24.668610 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:24.668780 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:24.672546 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:24.672552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:24.672555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:24.672591 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:24.675478 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:24.676476 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:24.676480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:24.676483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:24.676637 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:24.677139 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:24.677141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:24.677143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:24.677408 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:24.677518 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:24.678441 (+ 923us) server_negotiation.cc:299] Negotiation successful
0619 14:57:24.678621 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":297,"threads_started":1}
I20260619 14:57:24.680440 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:24.660159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:47059)
0619 14:57:24.660943 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:24.660998 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:24.662910 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:24.663785 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:24.663794 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:24.664451 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:24.665242 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:24.665264 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:24.666531 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:24.666535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:24.667922 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:24.667935 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:24.668884 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:24.668892 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:24.669148 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:24.669917 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:24.669940 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:24.672276 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:24.675660 (+ 3384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:24.675671 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:24.675692 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:24.676343 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:24.676756 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:24.676764 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:24.676770 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:24.677022 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:24.677536 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:24.677547 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:24.678031 (+ 484us) client_negotiation.cc:769] Sending connection context
0619 14:57:24.678337 (+ 306us) client_negotiation.cc:240] Negotiation successful
0619 14:57:24.678952 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":288,"threads_started":1}
I20260619 14:57:24.682011 20593 heartbeater.cc:347] Connected to a master server at 127.11.175.254:38869
I20260619 14:57:24.682374 20593 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:24.683195 20593 heartbeater.cc:511] Master 127.11.175.254:38869 requested a full tablet report, sending...
I20260619 14:57:24.684810 20270 ts_manager.cc:194] Registered new tserver with Master: 93f4cc31f9e44863a59c256a25f71015 (127.11.175.194:39739)
I20260619 14:57:24.685667 20270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:47059
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:57:24.719002 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38869
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:25.060405 20599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:25.060694 20599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:25.060873 20599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:25.068246 20599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:25.068383 20599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:25.068593 20599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:57:25.078600 20599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:25.079985 20599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:25.081650 20599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:25.096149 20605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:25.096151 20607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:25.096491 20604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:25.100622 20599 server_base.cc:1034] running on GCE node
I20260619 14:57:25.101502 20599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:25.102712 20599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:25.104031 20599 hybrid_clock.cc:648] HybridClock initialized: now 1781881045103984 us; error 59 us; skew 500 ppm
I20260619 14:57:25.104463 20599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:25 dist-test-slave-76rk krb5kdc[20225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881045, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:25.108147 20599 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:57:25.109925 20599 webserver.cc:460] Webserver started at http://127.11.175.195:36961/ using document root <none> and password file <none>
I20260619 14:57:25.110760 20599 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:25.110906 20599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:25.111227 20599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:25.114284 20599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "a88d848378204455880c1ba8f49ca495"
format_stamp: "Formatted at 2026-06-19 14:57:25 on dist-test-slave-76rk"
server_key: "2ed892015196b9194a6424f37ac5c525"
server_key_iv: "d12b8ff364a5160c99b5fce5322b160c"
server_key_version: "encryptionkey@0"
I20260619 14:57:25.115170 20599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "a88d848378204455880c1ba8f49ca495"
format_stamp: "Formatted at 2026-06-19 14:57:25 on dist-test-slave-76rk"
server_key: "2ed892015196b9194a6424f37ac5c525"
server_key_iv: "d12b8ff364a5160c99b5fce5322b160c"
server_key_version: "encryptionkey@0"
I20260619 14:57:25.120559 20599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:57:25.124163 20614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:25.125581 20599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 14:57:25.125715 20599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "a88d848378204455880c1ba8f49ca495"
format_stamp: "Formatted at 2026-06-19 14:57:25 on dist-test-slave-76rk"
server_key: "2ed892015196b9194a6424f37ac5c525"
server_key_iv: "d12b8ff364a5160c99b5fce5322b160c"
server_key_version: "encryptionkey@0"
I20260619 14:57:25.125890 20599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:25.146364 20459 heartbeater.cc:503] Master 127.11.175.254:38869 was elected leader, sending a full tablet report...
I20260619 14:57:25.161231 20599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:25.165998 20599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:25.166325 20599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:25.167428 20599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:25.168841 20599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:25.168920 20599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:25.169062 20599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:25.169121 20599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:25.204712 20599 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:42877
I20260619 14:57:25.204820 20726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:42877 every 8 connection(s)
I20260619 14:57:25.206072 20599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:57:25.213335 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 20599
I20260619 14:57:25.213526 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:57:25.213997 11967 external_mini_cluster.cc:1428] Setting key 04f2b82b7bbc9333604e0ed950efef0f
Jun 19 14:57:25 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881045, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:25.229611 20729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:25.208838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:46327 (local address 127.11.175.254:38869)
0619 14:57:25.209380 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:25.209385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:25.210397 (+ 1012us) server_negotiation.cc:407] Connection header received
0619 14:57:25.212549 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:25.212556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:25.212681 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:25.212822 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:25.214837 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:25.216085 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:25.218477 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:25.218985 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:25.219141 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:25.223173 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:25.223178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:25.223181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:25.223217 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:25.225894 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:25.226791 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:25.226796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:25.226798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:25.226926 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:25.227366 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:25.227369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:25.227371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:25.227637 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:25.227762 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:25.228804 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0619 14:57:25.229123 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":307,"threads_started":1}
I20260619 14:57:25.230845 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:25.209390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:46327)
0619 14:57:25.210198 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:25.210247 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:25.212234 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:25.213114 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:25.213124 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:25.213839 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:25.214579 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:25.214608 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:25.216299 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:25.216308 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:25.218312 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:25.218328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:25.219251 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:25.219256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:25.219489 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:25.220301 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:25.220329 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:25.222882 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:25.226069 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:25.226076 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:25.226091 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:25.226587 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:25.227047 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:25.227051 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:25.227055 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:25.227259 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:25.227788 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:25.227794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:25.228161 (+ 367us) client_negotiation.cc:769] Sending connection context
0619 14:57:25.228521 (+ 360us) client_negotiation.cc:240] Negotiation successful
0619 14:57:25.229144 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":339,"threads_started":1}
I20260619 14:57:25.232378 20727 heartbeater.cc:347] Connected to a master server at 127.11.175.254:38869
I20260619 14:57:25.232754 20727 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:25.233538 20727 heartbeater.cc:511] Master 127.11.175.254:38869 requested a full tablet report, sending...
I20260619 14:57:25.235184 20270 ts_manager.cc:194] Registered new tserver with Master: a88d848378204455880c1ba8f49ca495 (127.11.175.195:42877)
I20260619 14:57:25.236065 20270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:46327
I20260619 14:57:25.243930 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab.
Entry for principal oryx/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab.
WARNING: no policy specified for oryx/127.11.175.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab.
Entry for principal oryx/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab.
WARNING: no policy specified for oryx/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab.
Entry for principal oryx/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab.
I20260619 14:57:25.390201 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 20331
Jun 19 14:57:25 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:57:25.410892 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:25.391713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35552 (local address 127.11.175.193:37811)
0619 14:57:25.392549 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:25.392557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:25.392588 (+ 31us) server_negotiation.cc:407] Connection header received
0619 14:57:25.392653 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:25.392660 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:25.392906 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:25.393098 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:25.394760 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:25.396071 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:25.397578 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:25.398249 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:25.398376 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:25.403348 (+ 4972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:25.403365 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:25.403381 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:25.403423 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:25.407122 (+ 3699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:25.408181 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:25.408189 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:25.408191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:25.408363 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:25.408944 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:25.408950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:25.408952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:25.409425 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:25.409691 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:25.410069 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 14:57:25.410387 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":412,"threads_started":1}
I20260619 14:57:25.412370 20454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:25.496791 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 20331
I20260619 14:57:25.513761 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 20465
Jun 19 14:57:25 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:57:25.531344 20740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:25.514727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52336 (local address 127.11.175.194:39739)
0619 14:57:25.515354 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:25.515361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:25.515387 (+ 26us) server_negotiation.cc:407] Connection header received
0619 14:57:25.515456 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:25.515462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:25.515725 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:25.515876 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:25.517069 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:25.518057 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:25.519390 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:25.519934 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:25.520031 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:25.524614 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:25.524632 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:25.524651 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:25.524709 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:25.527764 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:25.528852 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:25.528865 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:25.528871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:25.529122 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:25.529575 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:25.529579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:25.529582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:25.530091 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:25.530401 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:25.530828 (+ 427us) server_negotiation.cc:299] Negotiation successful
0619 14:57:25.531055 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":344,"threads_started":1}
I20260619 14:57:25.532706 20588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:25.617756 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 20465
I20260619 14:57:25.634061 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 20599
Jun 19 14:57:25 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:57:25.651654 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:25.635113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49470 (local address 127.11.175.195:42877)
0619 14:57:25.635601 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:25.635608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:25.635656 (+ 48us) server_negotiation.cc:407] Connection header received
0619 14:57:25.635715 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:25.635721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:25.635926 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:25.636122 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:25.637478 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:25.638484 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:25.639885 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:25.640450 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:25.640606 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:25.645488 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:25.645503 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:25.645519 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:25.645565 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:25.648670 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:25.649430 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:25.649438 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:25.649440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:25.649578 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:25.649981 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:25.649992 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:25.649995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:25.650366 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:25.650719 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:25.651035 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 14:57:25.651302 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":223,"threads_started":1}
I20260619 14:57:25.653131 20722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:25.736683 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 20599
I20260619 14:57:25.753661 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 20238
I20260619 14:57:25.754947 20300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:25.859364 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 20238
I20260619 14:57:25.880918 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38869
--webserver_interface=127.11.175.254
--webserver_port=40895
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:26.226854 20745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:26.227308 20745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:26.227500 20745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:26.235551 20745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:26.235737 20745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:26.235852 20745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:26.235962 20745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:26.236048 20745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:26.246313 20745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=40895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:26.247736 20745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:26.249046 20745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:26.261185 20750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:26.261307 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:26.261299 20753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:26.262459 20745 server_base.cc:1034] running on GCE node
I20260619 14:57:26.263428 20745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:26.265122 20745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:26.266359 20745 hybrid_clock.cc:648] HybridClock initialized: now 1781881046266329 us; error 56 us; skew 500 ppm
I20260619 14:57:26.266793 20745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:26 dist-test-slave-76rk krb5kdc[20225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881046, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:26.270053 20745 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:57:26.271643 20745 webserver.cc:460] Webserver started at http://127.11.175.254:40895/ using document root <none> and password file <none>
I20260619 14:57:26.272325 20745 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:26.272441 20745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:26.278277 20745 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260619 14:57:26.281461 20760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:26.283042 20745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 14:57:26.283234 20745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "d51711a8ea134400b1fa3af77194b389"
format_stamp: "Formatted at 2026-06-19 14:57:23 on dist-test-slave-76rk"
server_key: "0638988287f0b9583112680c0a4e1480"
server_key_iv: "d74125037bcc7ad2deabc6cb923b1fe9"
server_key_version: "encryptionkey@0"
I20260619 14:57:26.284335 20745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:26.321633 20745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:26.326426 20745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:26.326771 20745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:26.346570 20745 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:38869
I20260619 14:57:26.346607 20811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:38869 every 8 connection(s)
I20260619 14:57:26.348179 20745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:57:26.355125 20812 sys_catalog.cc:263] Verifying existing consensus state
I20260619 14:57:26.355959 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 20745
I20260619 14:57:26.356783 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:37811
--local_ip_for_outbound_sockets=127.11.175.193
--tserver_master_addrs=127.11.175.254:38869
--webserver_port=36193
--webserver_interface=127.11.175.193
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.175.193
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:57:26.358807 20812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389: Bootstrap starting.
I20260619 14:57:26.384153 20812 log.cc:824] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:26.396471 20812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 14:57:26.397192 20812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389: Bootstrap complete.
I20260619 14:57:26.411638 20812 raft_consensus.cc:348] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } }
I20260619 14:57:26.412886 20812 raft_consensus.cc:729] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51711a8ea134400b1fa3af77194b389, State: Initialized, Role: FOLLOWER
I20260619 14:57:26.413717 20812 consensus_queue.cc:260] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } }
I20260619 14:57:26.414019 20812 raft_consensus.cc:388] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:57:26.414156 20812 raft_consensus.cc:482] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:57:26.414341 20812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 1 FOLLOWER]: Advancing to term 2
I20260619 14:57:26.418926 20812 raft_consensus.cc:504] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } }
I20260619 14:57:26.419535 20812 leader_election.cc:302] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d51711a8ea134400b1fa3af77194b389; no voters:
I20260619 14:57:26.420168 20812 leader_election.cc:288] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 14:57:26.420384 20816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 14:57:26.421622 20816 raft_consensus.cc:686] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [term 2 LEADER]: Becoming Leader. State: Replica: d51711a8ea134400b1fa3af77194b389, State: Running, Role: LEADER
I20260619 14:57:26.422469 20812 sys_catalog.cc:564] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:57:26.422343 20816 consensus_queue.cc:237] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } }
I20260619 14:57:26.425419 20817 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d51711a8ea134400b1fa3af77194b389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } } }
I20260619 14:57:26.425655 20818 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d51711a8ea134400b1fa3af77194b389. Latest consensus state: current_term: 2 leader_uuid: "d51711a8ea134400b1fa3af77194b389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51711a8ea134400b1fa3af77194b389" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38869 } } }
I20260619 14:57:26.426074 20817 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:26.426082 20818 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:26.427225 20824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:57:26.443066 20824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:57:26.444783 20824 catalog_manager.cc:1259] Loaded cluster ID: a38aba547970426da91d0160e831a90b
I20260619 14:57:26.444952 20824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:57:26.449234 20824 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:57:26.450871 20824 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d51711a8ea134400b1fa3af77194b389: Loaded TSK: 0
I20260619 14:57:26.452502 20824 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 14:57:26.740485 20814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:26.740841 20814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:26.741014 20814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:26.748951 20814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:26.749112 20814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:26.749301 20814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:57:26.760056 20814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.193.keytab
--principal=oryx/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:37811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=36193
--tserver_master_addrs=127.11.175.254:38869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:26.761464 20814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:26.762946 20814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:26.778244 20838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:26.778319 20839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:26.779557 20841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:26.780547 20814 server_base.cc:1034] running on GCE node
I20260619 14:57:26.781375 20814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:26.782770 20814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:26.784085 20814 hybrid_clock.cc:648] HybridClock initialized: now 1781881046783992 us; error 71 us; skew 500 ppm
I20260619 14:57:26.784482 20814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:26 dist-test-slave-76rk krb5kdc[20225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881046, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:26.788813 20814 init.cc:377] Logged in from keytab as oryx/127.11.175.193@KRBTEST.COM (short username oryx)
I20260619 14:57:26.790751 20814 webserver.cc:460] Webserver started at http://127.11.175.193:36193/ using document root <none> and password file <none>
I20260619 14:57:26.791672 20814 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:26.791829 20814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:26.799115 20814 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 14:57:26.802625 20848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:26.804526 20814 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 14:57:26.804718 20814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "79743877f62d4ed4aa91a2ed31ed5343"
format_stamp: "Formatted at 2026-06-19 14:57:24 on dist-test-slave-76rk"
server_key: "fd9a9f8998b71d4843e06726e434494c"
server_key_iv: "48a78bd751cfaa8b29b8c34811be7c58"
server_key_version: "encryptionkey@0"
I20260619 14:57:26.806095 20814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:26.828564 20814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:26.832746 20814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:26.832978 20814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:26.833923 20814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:26.835405 20814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:26.835493 20814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:26.835584 20814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:26.835661 20814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:26.873548 20814 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:37811
I20260619 14:57:26.873646 20960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:37811 every 8 connection(s)
I20260619 14:57:26.875226 20814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:57:26.876839 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 20814
I20260619 14:57:26.877617 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:39739
--local_ip_for_outbound_sockets=127.11.175.194
--tserver_master_addrs=127.11.175.254:38869
--webserver_port=36659
--webserver_interface=127.11.175.194
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.175.194
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 14:57:26 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:26 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:26.896934 20964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:26.879119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:52437)
0619 14:57:26.880391 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:26.880440 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:26.882421 (+ 1981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:26.884396 (+ 1975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:26.884405 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:26.884982 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:26.885605 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:26.885623 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.887501 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.887507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:26.889018 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:26.889033 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.890207 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.890215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:26.890508 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:26.891322 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:26.891352 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:26.895513 (+ 4161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":978,"thread_start_us":754,"threads_started":1}
W20260619 14:57:26.897086 20963 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:26.879101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:52437 (local address 127.11.175.254:38869)
0619 14:57:26.879951 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:26.879969 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:26.880621 (+ 652us) server_negotiation.cc:407] Connection header received
0619 14:57:26.883194 (+ 2573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:26.883217 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:26.883735 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:26.884216 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:26.885865 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.887293 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.889204 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.889889 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.890152 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:26.895963 (+ 5811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:52437: BlockingRecv error: recv got EOF from 127.11.175.193:52437 (error 108)
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":321,"threads_started":1}
W20260619 14:57:26.897744 20961 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:38869 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.175.254:38869: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Jun 19 14:57:26 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:26 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:26.909224 20964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:26.898408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:43979)
0619 14:57:26.898689 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:26.898717 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:26.898863 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:26.899290 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:26.899298 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:26.899827 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:26.900098 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:26.900114 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.901197 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.901205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:26.903061 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:26.903083 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.903857 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.903867 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:26.904119 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:26.904896 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:26.904925 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:26.908872 (+ 3947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260619 14:57:26.909543 20963 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:26.898609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:43979 (local address 127.11.175.254:38869)
0619 14:57:26.898812 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:26.898817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:26.898834 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:57:26.899020 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:26.899024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:26.899127 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:26.899204 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:26.900283 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.901021 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.903262 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.903679 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.903779 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:26.909255 (+ 5476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:43979: BlockingRecv error: recv got EOF from 127.11.175.193:43979 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Jun 19 14:57:26 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:26 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:26.920396 20964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:26.910083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:46643)
0619 14:57:26.910312 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:26.910333 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:26.910463 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:26.910936 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:26.910942 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:26.911400 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:26.911662 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:26.911680 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.912877 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.912883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:26.914516 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:26.914536 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.915346 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.915354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:26.915594 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:26.916417 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:26.916447 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:26.920073 (+ 3626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
W20260619 14:57:26.920579 20963 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:26.910221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:46643 (local address 127.11.175.254:38869)
0619 14:57:26.910368 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:26.910372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:26.910389 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:57:26.910625 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:26.910629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:26.910787 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:26.910877 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:26.911833 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.912718 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.914776 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:26.915187 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:26.915287 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:26.920323 (+ 5036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:46643: BlockingRecv error: recv got EOF from 127.11.175.193:46643 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260619 14:57:26.920877 20961 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 14:57:27.227090 20965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:27.227380 20965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:27.227478 20965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:27.234680 20965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:27.234794 20965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:27.234905 20965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:57:27.244753 20965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.194.keytab
--principal=oryx/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:39739
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=36659
--tserver_master_addrs=127.11.175.254:38869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:27.246155 20965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:27.247526 20965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:27.261814 20973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:27.261834 20971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:27.261839 20970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:27.262542 20965 server_base.cc:1034] running on GCE node
I20260619 14:57:27.263370 20965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:27.264348 20965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:27.265627 20965 hybrid_clock.cc:648] HybridClock initialized: now 1781881047265564 us; error 57 us; skew 500 ppm
I20260619 14:57:27.265980 20965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881047, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:27.269423 20965 init.cc:377] Logged in from keytab as oryx/127.11.175.194@KRBTEST.COM (short username oryx)
I20260619 14:57:27.271211 20965 webserver.cc:460] Webserver started at http://127.11.175.194:36659/ using document root <none> and password file <none>
I20260619 14:57:27.271988 20965 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:27.272169 20965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:27.278075 20965 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260619 14:57:27.281178 20980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:27.282766 20965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:57:27.282994 20965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "93f4cc31f9e44863a59c256a25f71015"
format_stamp: "Formatted at 2026-06-19 14:57:24 on dist-test-slave-76rk"
server_key: "39e0bcfa4e6fcad6d07f3f8e74dd0875"
server_key_iv: "169e9fc6d4cd5a81263116a7120813e8"
server_key_version: "encryptionkey@0"
I20260619 14:57:27.284140 20965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:27.315073 20965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:27.319423 20965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:27.319749 20965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:27.320775 20965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:27.322182 20965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:27.322341 20965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:27.322476 20965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:27.322566 20965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:27.358270 20965 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:39739
I20260619 14:57:27.358312 21092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:39739 every 8 connection(s)
I20260619 14:57:27.359866 20965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:57:27.365521 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 20965
I20260619 14:57:27.366246 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:42877
--local_ip_for_outbound_sockets=127.11.175.195
--tserver_master_addrs=127.11.175.254:38869
--webserver_port=36961
--webserver_interface=127.11.175.195
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.175.195
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 14:57:27.377771 20963 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:27.362785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:37623 (local address 127.11.175.254:38869)
0619 14:57:27.362999 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:27.363004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:27.364193 (+ 1189us) server_negotiation.cc:407] Connection header received
0619 14:57:27.366097 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:27.366106 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:27.366211 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:27.366341 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:27.368023 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.369151 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.371018 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.371812 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.371966 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.377456 (+ 5490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:37623: BlockingRecv error: recv got EOF from 127.11.175.194:37623 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260619 14:57:27.378280 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:27.363320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:37623)
0619 14:57:27.364014 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:27.364061 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:27.365817 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:27.366712 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:27.366721 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:27.367243 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:27.367824 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.367842 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.369369 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.369374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.370835 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.370848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.372120 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.372126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.372349 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.373146 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:27.373170 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:27.377174 (+ 4004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":281,"threads_started":1}
W20260619 14:57:27.379022 21093 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:38869 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.175.254:38869: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:27.390434 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:27.379684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:40015)
0619 14:57:27.379977 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:27.379999 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:27.380112 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:27.380469 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:27.380477 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:27.380942 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:27.381172 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.381186 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.382286 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.382294 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.384037 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.384056 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.384861 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.384870 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.385132 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.385812 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:27.385852 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:27.390043 (+ 4191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139}
W20260619 14:57:27.390574 20963 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:27.379859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:40015 (local address 127.11.175.254:38869)
0619 14:57:27.380036 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:27.380040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:27.380057 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:57:27.380235 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:27.380239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:27.380301 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:27.380387 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:27.381330 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.382111 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.384200 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.384619 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.384815 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.390306 (+ 5491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:40015: BlockingRecv error: recv got EOF from 127.11.175.194:40015 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:27.400573 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:27.391341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:52761)
0619 14:57:27.391548 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:27.391567 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:27.391683 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:27.392077 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:27.392080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:27.392362 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:27.392543 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.392551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.394159 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.394163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.395315 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.395328 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.396287 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.396292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.396423 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.397089 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:27.397110 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:27.400325 (+ 3215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260619 14:57:27.400847 20963 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:27.391440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:52761 (local address 127.11.175.254:38869)
0619 14:57:27.391643 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:27.391652 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:27.391677 (+ 25us) server_negotiation.cc:407] Connection header received
0619 14:57:27.391794 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:27.391802 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:27.391895 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:27.392035 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:27.392700 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.394006 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.395452 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.396120 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.396243 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.400614 (+ 4371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:52761: BlockingRecv error: recv got EOF from 127.11.175.194:52761 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 14:57:27.401243 21093 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 14:57:27.709406 21096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:27.709717 21096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:27.709895 21096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:27.717943 21096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:27.718068 21096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:27.718281 21096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:57:27.728833 21096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/krb5kdc/oryx_127.11.175.195.keytab
--principal=oryx/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:42877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=36961
--tserver_master_addrs=127.11.175.254:38869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:27.730183 21096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:27.731618 21096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:27.745189 21101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:27.745198 21102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:27.745433 21104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:27.746094 21096 server_base.cc:1034] running on GCE node
I20260619 14:57:27.746752 21096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:27.747844 21096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:27.749116 21096 hybrid_clock.cc:648] HybridClock initialized: now 1781881047749047 us; error 59 us; skew 500 ppm
I20260619 14:57:27.749466 21096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881047, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:27.752980 21096 init.cc:377] Logged in from keytab as oryx/127.11.175.195@KRBTEST.COM (short username oryx)
I20260619 14:57:27.754680 21096 webserver.cc:460] Webserver started at http://127.11.175.195:36961/ using document root <none> and password file <none>
I20260619 14:57:27.755443 21096 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:27.755615 21096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:27.761672 21096 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 14:57:27.765008 21111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:27.766597 21096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 14:57:27.766877 21096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "a88d848378204455880c1ba8f49ca495"
format_stamp: "Formatted at 2026-06-19 14:57:25 on dist-test-slave-76rk"
server_key: "2ed892015196b9194a6424f37ac5c525"
server_key_iv: "d12b8ff364a5160c99b5fce5322b160c"
server_key_version: "encryptionkey@0"
I20260619 14:57:27.767911 21096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:27.781040 21096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:27.785297 21096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:27.785593 21096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:27.786597 21096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:27.788024 21096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:27.788126 21096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:27.788198 21096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:27.788254 21096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:27.822894 21096 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:42877
I20260619 14:57:27.822974 21223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:42877 every 8 connection(s)
I20260619 14:57:27.824452 21096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:57:27.834101 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 21096
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 14:57:27.840855 20963 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:27.827154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:47139 (local address 127.11.175.254:38869)
0619 14:57:27.827373 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:27.827379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:27.828550 (+ 1171us) server_negotiation.cc:407] Connection header received
0619 14:57:27.830359 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:27.830363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:27.830445 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:27.830538 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:27.832109 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.832860 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.834687 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.835144 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.835310 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.840544 (+ 5234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:47139: BlockingRecv error: recv got EOF from 127.11.175.195:47139 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 14:57:27.841341 21226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:27.827676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:47139)
0619 14:57:27.828374 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:27.828416 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:27.830091 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:27.830846 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:27.830855 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:27.831374 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:27.831942 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.831959 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.833074 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.833078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.834495 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.834509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.835478 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.835492 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.835883 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.836667 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:27.836690 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:27.840273 (+ 3583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":271,"threads_started":1}
W20260619 14:57:27.842197 21224 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:38869 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.175.254:38869: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:27.852238 21226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:27.843075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:43677)
0619 14:57:27.843333 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:27.843354 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:27.843468 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:27.843805 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:27.843812 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:27.844336 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:27.844533 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.844547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.845570 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.845577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.846771 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.846784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.847905 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.847911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.848120 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.848717 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:27.848740 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:27.851912 (+ 3172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142}
W20260619 14:57:27.852420 20963 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:27.843192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:43677 (local address 127.11.175.254:38869)
0619 14:57:27.843347 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:27.843351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:27.843406 (+ 55us) server_negotiation.cc:407] Connection header received
0619 14:57:27.843581 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:27.843584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:27.843635 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:27.843714 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:27.844701 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.845412 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.846918 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.847750 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.847879 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.852135 (+ 4256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:43677: BlockingRecv error: recv got EOF from 127.11.175.195:43677 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 14:57:27.853085 21227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:27.835299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55756 (local address 127.11.175.254:38869)
0619 14:57:27.836650 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:27.836655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:27.836671 (+ 16us) server_negotiation.cc:407] Connection header received
0619 14:57:27.836716 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:27.836719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:27.836785 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:27.836933 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:27.838665 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.839554 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.841286 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.841912 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.842064 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.845402 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:27.845411 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:27.845429 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:27.845464 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:27.849208 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:27.850179 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:27.850189 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:27.850203 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:27.850398 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:27.850876 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:27.850883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:27.850887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:27.851436 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:27.851758 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:27.852236 (+ 478us) server_negotiation.cc:299] Negotiation successful
0619 14:57:27.852759 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1244,"thread_start_us":1092,"threads_started":1}
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:27.863317 21226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:27.853569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:32987)
0619 14:57:27.853745 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:27.853763 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:27.853893 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:27.854288 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:27.854293 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:27.854765 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:27.854968 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.854981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.856890 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.856897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.858064 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.858077 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.859047 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.859051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.859187 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.859818 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:27.859835 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:27.863065 (+ 3230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 14:57:27.863565 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:27.853703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:32987 (local address 127.11.175.254:38869)
0619 14:57:27.854025 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:27.854034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:27.854047 (+ 13us) server_negotiation.cc:407] Connection header received
0619 14:57:27.854106 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:27.854109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:27.854157 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:27.854260 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:27.855159 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.856633 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.858224 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.858883 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.859006 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.863289 (+ 4283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:32987: BlockingRecv error: recv got EOF from 127.11.175.195:32987 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
W20260619 14:57:27.863741 21224 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:27 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:27.930263 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:27.921516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:58667)
0619 14:57:27.922141 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:27.922157 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:27.922319 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:27.922615 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:27.922618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:27.923165 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:27.923422 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.923433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.924538 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.924542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.925629 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:27.925643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.926349 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.926354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:27.926481 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.927075 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:27.927099 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:27.930019 (+ 2920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":351,"threads_started":1}
W20260619 14:57:27.930716 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:27.921735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:58667 (local address 127.11.175.254:38869)
0619 14:57:27.921987 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:27.921996 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:27.922270 (+ 274us) server_negotiation.cc:407] Connection header received
0619 14:57:27.922424 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:27.922427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:27.922488 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:27.922570 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:27.923551 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.924363 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.925771 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:27.926172 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:27.926264 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:27.930283 (+ 4019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:58667: BlockingRecv error: recv got EOF from 127.11.175.193:58667 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Jun 19 14:57:28 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:28 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:28.410310 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:28.401773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:56905)
0619 14:57:28.402264 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:28.402284 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:28.402394 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:28.402750 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:28.402754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:28.403295 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:28.403492 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:28.403502 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.404574 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.404578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:28.405597 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:28.405609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.406326 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.406331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:28.406457 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:28.407052 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:28.407069 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:28.409989 (+ 2920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":296,"threads_started":1}
W20260619 14:57:28.410408 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:28.401919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:56905 (local address 127.11.175.254:38869)
0619 14:57:28.402169 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:28.402175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:28.402369 (+ 194us) server_negotiation.cc:407] Connection header received
0619 14:57:28.402492 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:28.402496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:28.402557 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:28.402726 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:28.403639 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.404412 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.405734 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.406144 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.406238 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:28.410212 (+ 3974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:56905: BlockingRecv error: recv got EOF from 127.11.175.194:56905 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Jun 19 14:57:28 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:28 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:28.876380 21231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:28.864363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:52365)
0619 14:57:28.865156 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:28.865174 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:28.865301 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:28.865695 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:28.865700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:28.866225 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:28.866422 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:28.866433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.868146 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.868154 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:28.869840 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:28.869855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.870978 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.870985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:28.871174 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:28.872734 (+ 1560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:28.872758 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:28.876150 (+ 3392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":305,"threads_started":1}
W20260619 14:57:28.876649 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:28.864595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:52365 (local address 127.11.175.254:38869)
0619 14:57:28.864881 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:28.864890 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:28.865264 (+ 374us) server_negotiation.cc:407] Connection header received
0619 14:57:28.865473 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:28.865477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:28.865540 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:28.865660 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:28.866586 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.867965 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.870003 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.870786 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.870916 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:28.876385 (+ 5469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:52365: BlockingRecv error: recv got EOF from 127.11.175.195:52365 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Jun 19 14:57:28 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:28 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:28.939953 21232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:28.931302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:56937)
0619 14:57:28.931875 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:28.931893 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:28.932123 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:28.932524 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:28.932528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:28.933083 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:28.933339 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:28.933356 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.934487 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.934491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:28.935421 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:28.935433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.936154 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.936158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:28.936264 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:28.936821 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:28.936839 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:28.939714 (+ 2875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":336,"threads_started":1}
W20260619 14:57:28.940169 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:28.931408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:56937 (local address 127.11.175.254:38869)
0619 14:57:28.931608 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:28.931612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:28.932014 (+ 402us) server_negotiation.cc:407] Connection header received
0619 14:57:28.932275 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:28.932280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:28.932369 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:28.932474 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:28.933501 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.934361 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.935554 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:28.936000 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:28.936109 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:28.939918 (+ 3809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:56937: BlockingRecv error: recv got EOF from 127.11.175.193:56937 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 14:57:29 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:29 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:29.419888 21233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:29.411351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:35409)
0619 14:57:29.411896 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:29.411915 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:29.412093 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:29.412444 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:29.412448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:29.412858 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:29.413066 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:29.413075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.414093 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.414096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:29.415021 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:29.415032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.415820 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.415826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:29.415989 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:29.416722 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:29.416739 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:29.419655 (+ 2916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":325,"threads_started":1}
W20260619 14:57:29.420459 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:29.411436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:35409 (local address 127.11.175.254:38869)
0619 14:57:29.411658 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:29.411665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:29.412043 (+ 378us) server_negotiation.cc:407] Connection header received
0619 14:57:29.412201 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:29.412205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:29.412271 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:29.412355 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:29.413219 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.413965 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.415153 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.415660 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.415760 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:29.420240 (+ 4480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:35409: BlockingRecv error: recv got EOF from 127.11.175.194:35409 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 19 14:57:29 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:29 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:29.891021 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:29.877409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:59117)
0619 14:57:29.877892 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:29.877909 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:29.878019 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:29.878334 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:29.878338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:29.878867 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:29.879040 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:29.879049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.879996 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.880000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:29.880890 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:29.880901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.881570 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.881587 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:29.881700 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:29.887459 (+ 5759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:29.887481 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:29.890638 (+ 3157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":264,"threads_started":1}
W20260619 14:57:29.891127 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:29.877557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:59117 (local address 127.11.175.254:38869)
0619 14:57:29.877805 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:29.877811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:29.877995 (+ 184us) server_negotiation.cc:407] Connection header received
0619 14:57:29.878136 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:29.878140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:29.878214 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:29.878293 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:29.879167 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.879888 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.881018 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.881436 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.881531 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:29.890892 (+ 9361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:59117: BlockingRecv error: recv got EOF from 127.11.175.195:59117 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Jun 19 14:57:29 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:29 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:29.949636 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:29.940977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:40883)
0619 14:57:29.941566 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:29.941588 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:29.941705 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:29.942047 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:29.942050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:29.942390 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:29.942551 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:29.942559 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.943622 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.943626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:29.944508 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:29.944519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.945252 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.945257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:29.945387 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:29.946043 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:29.946062 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:29.949321 (+ 3259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":347,"threads_started":1}
W20260619 14:57:29.949736 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:29.941104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:40883 (local address 127.11.175.254:38869)
0619 14:57:29.941293 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:29.941298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:29.941668 (+ 370us) server_negotiation.cc:407] Connection header received
0619 14:57:29.941849 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:29.941853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:29.941918 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:29.942004 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:29.942755 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.943467 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.944667 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:29.945094 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:29.945196 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:29.949503 (+ 4307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:40883: BlockingRecv error: recv got EOF from 127.11.175.193:40883 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 14:57:30 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:30 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:30.429457 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:30.420862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:44465)
0619 14:57:30.421401 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:30.421422 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:30.421540 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:30.421905 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:30.421909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:30.422444 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:30.422764 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:30.422776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.424238 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.424242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:30.425150 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:30.425161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.425818 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.425822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:30.425940 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:30.426510 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:30.426526 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:30.429201 (+ 2675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":293,"threads_started":1}
W20260619 14:57:30.429597 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:30.420950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:44465 (local address 127.11.175.254:38869)
0619 14:57:30.421142 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:30.421146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:30.421509 (+ 363us) server_negotiation.cc:407] Connection header received
0619 14:57:30.421643 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:30.421646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:30.421709 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:30.421830 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:30.422911 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.424096 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.425282 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.425699 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.425795 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:30.429385 (+ 3590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:44465: BlockingRecv error: recv got EOF from 127.11.175.194:44465 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 19 14:57:30 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:30 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:30.901090 21237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:30.892037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:40127)
0619 14:57:30.892581 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:30.892597 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:30.892720 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:30.893073 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:30.893076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:30.893550 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:30.893746 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:30.893756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.894994 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.895001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:30.896003 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:30.896015 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.896721 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.896727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:30.896910 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:30.897494 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:30.897514 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:30.900708 (+ 3194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":316,"threads_started":1}
W20260619 14:57:30.901225 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:30.892126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:40127 (local address 127.11.175.254:38869)
0619 14:57:30.892320 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:30.892325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:30.892687 (+ 362us) server_negotiation.cc:407] Connection header received
0619 14:57:30.892826 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:30.892830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:30.892912 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:30.893084 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:30.893913 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.894851 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.896154 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.896564 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.896693 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:30.900949 (+ 4256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:40127: BlockingRecv error: recv got EOF from 127.11.175.195:40127 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Jun 19 14:57:30 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:30 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:30.960053 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:30.950737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:58039)
0619 14:57:30.951271 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:30.951287 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:30.951455 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:30.951868 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:30.951873 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:30.952296 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:30.952451 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:30.952461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.953515 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.953522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:30.954528 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:30.954541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.955365 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.955371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:30.955566 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:30.956264 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:30.956286 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:30.959713 (+ 3427us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":300,"threads_started":1}
W20260619 14:57:30.960139 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:30.950842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:58039 (local address 127.11.175.254:38869)
0619 14:57:30.951106 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:30.951113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:30.951375 (+ 262us) server_negotiation.cc:407] Connection header received
0619 14:57:30.951603 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:30.951608 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:30.951685 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:30.951811 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:30.952600 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.953366 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.954693 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:30.955215 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:30.955352 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:30.959921 (+ 4569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:58039: BlockingRecv error: recv got EOF from 127.11.175.193:58039 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Jun 19 14:57:31 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:31 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:31.439481 21239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:31.430678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:49523)
0619 14:57:31.431322 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:31.431341 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:31.431506 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:31.431872 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:31.431876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:31.432274 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:31.432420 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:31.432428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.433471 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.433478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:31.434510 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:31.434522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.435646 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.435650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:31.435791 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:31.436443 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:31.436460 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:31.439249 (+ 2789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":389,"threads_started":1}
W20260619 14:57:31.439810 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:31.430691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:49523 (local address 127.11.175.254:38869)
0619 14:57:31.430931 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:31.430940 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:31.431438 (+ 498us) server_negotiation.cc:407] Connection header received
0619 14:57:31.431638 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:31.431644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:31.431717 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:31.431822 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:31.432568 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.433335 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.434754 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.435512 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.435616 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:31.439516 (+ 3900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:49523: BlockingRecv error: recv got EOF from 127.11.175.194:49523 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 19 14:57:31 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:31 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:31.911191 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:31.902216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:35461)
0619 14:57:31.902810 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:31.902832 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:31.902988 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:31.903356 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:31.903360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:31.903793 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:31.903938 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:31.903947 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.904949 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.904955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:31.905914 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:31.905925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.906687 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.906693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:31.906870 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:31.907456 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:31.907474 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:31.910844 (+ 3370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":313,"threads_started":1}
W20260619 14:57:31.911405 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:31.902372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:35461 (local address 127.11.175.254:38869)
0619 14:57:31.902549 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:31.902554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:31.902901 (+ 347us) server_negotiation.cc:407] Connection header received
0619 14:57:31.903129 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:31.903134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:31.903218 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:31.903326 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:31.904113 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.904815 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.906059 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.906500 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.906634 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:31.911134 (+ 4500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:35461: BlockingRecv error: recv got EOF from 127.11.175.195:35461 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 14:57:31 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:31 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:31.969674 21241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:31.960986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:59129)
0619 14:57:31.961694 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:31.961710 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:31.961877 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:31.962275 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:31.962281 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:31.962767 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:31.962930 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:31.962939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.963954 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.963962 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:31.964904 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:31.964916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.965659 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.965665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:31.965847 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:31.966474 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:31.966492 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:31.969304 (+ 2812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":281,"threads_started":1}
W20260619 14:57:31.969810 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:31.961180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:59129 (local address 127.11.175.254:38869)
0619 14:57:31.961441 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:31.961450 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:31.961809 (+ 359us) server_negotiation.cc:407] Connection header received
0619 14:57:31.962061 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:31.962066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:31.962129 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:31.962239 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:31.963091 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.963820 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.965063 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:31.965498 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:31.965646 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:31.969544 (+ 3898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:59129: BlockingRecv error: recv got EOF from 127.11.175.193:59129 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Jun 19 14:57:32 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:32 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:32.449047 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:32.440489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:41711)
0619 14:57:32.441372 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:32.441402 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:32.441544 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:32.441902 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:32.441905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:32.442282 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:32.442441 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:32.442448 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.443496 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.443500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:32.444421 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:32.444437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.445143 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.445147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:32.445266 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:32.445924 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:32.445942 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:32.448744 (+ 2802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":524,"threads_started":1}
W20260619 14:57:32.449152 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:32.440655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:41711 (local address 127.11.175.254:38869)
0619 14:57:32.440903 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:32.440913 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:32.441491 (+ 578us) server_negotiation.cc:407] Connection header received
0619 14:57:32.441673 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:32.441678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:32.441776 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:32.441855 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:32.442562 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.443375 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.444554 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.444990 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.445084 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:32.448926 (+ 3842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:41711: BlockingRecv error: recv got EOF from 127.11.175.194:41711 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Jun 19 14:57:32 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:32 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:32.921066 21243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:32.912244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:57513)
0619 14:57:32.912839 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:32.912857 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:32.913047 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:32.913443 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:32.913448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:32.913991 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:32.914168 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:32.914177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.915316 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.915323 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:32.916253 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:32.916264 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.916994 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.916999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:32.917194 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:32.917814 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:32.917832 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:32.920691 (+ 2859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":301,"threads_started":1}
W20260619 14:57:32.921209 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:32.912384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:57513 (local address 127.11.175.254:38869)
0619 14:57:32.912585 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:32.912606 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:32.912973 (+ 367us) server_negotiation.cc:407] Connection header received
0619 14:57:32.913198 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:32.913204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:32.913295 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:32.913414 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:32.914317 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.915176 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.916400 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.916839 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.916971 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:32.920923 (+ 3952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:57513: BlockingRecv error: recv got EOF from 127.11.175.195:57513 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Jun 19 14:57:32 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:32 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:32.979786 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:32.970884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:45371)
0619 14:57:32.971508 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:32.971526 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:32.971660 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:32.972197 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:32.972201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:32.972605 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:32.972763 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:32.972773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.973875 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.973879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:32.974897 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:32.974909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.975728 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.975733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:32.975909 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:32.976560 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:32.976579 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:32.979560 (+ 2981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":352,"threads_started":1}
W20260619 14:57:32.980082 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:32.970916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:45371 (local address 127.11.175.254:38869)
0619 14:57:32.971151 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:32.971157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:32.971605 (+ 448us) server_negotiation.cc:407] Connection header received
0619 14:57:32.971777 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:32.971801 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:32.971884 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:32.971983 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:32.973000 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.973729 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.975063 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:32.975529 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:32.975685 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:32.979799 (+ 4114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:45371: BlockingRecv error: recv got EOF from 127.11.175.193:45371 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 14:57:33 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:33 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:33.458515 21245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:33.449992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:51911)
0619 14:57:33.450567 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:33.450589 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:33.450785 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:33.451154 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:33.451160 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:33.451585 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:33.451734 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:33.451742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.452791 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.452796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:33.453712 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:33.453724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.454443 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.454449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:33.454670 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:33.455325 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:33.455344 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:33.458271 (+ 2927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":304,"threads_started":1}
W20260619 14:57:33.459022 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:33.450166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:51911 (local address 127.11.175.254:38869)
0619 14:57:33.450432 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:33.450438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:33.450689 (+ 251us) server_negotiation.cc:407] Connection header received
0619 14:57:33.450911 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:33.450917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:33.450992 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:33.451107 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:33.451882 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.452629 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.453858 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.454276 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.454405 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:33.458650 (+ 4245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:51911: BlockingRecv error: recv got EOF from 127.11.175.194:51911 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Jun 19 14:57:33 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:33 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:33.931269 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:33.922145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:53997)
0619 14:57:33.922728 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:33.922745 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:33.922908 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:33.923296 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:33.923300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:33.923712 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:33.923859 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:33.923867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.925387 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.925391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:33.926613 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:33.926626 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.927529 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.927534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:33.927675 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:33.928274 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:33.928293 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:33.931055 (+ 2762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":315,"threads_started":1}
W20260619 14:57:33.931461 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:33.922286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:53997 (local address 127.11.175.254:38869)
0619 14:57:33.922498 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:33.922504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:33.922875 (+ 371us) server_negotiation.cc:407] Connection header received
0619 14:57:33.923021 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:33.923025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:33.923126 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:33.923216 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:33.924167 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.925236 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.926800 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.927372 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.927480 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:33.931241 (+ 3761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:53997: BlockingRecv error: recv got EOF from 127.11.175.195:53997 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 14:57:33 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:33 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:33.990078 21247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:33.980824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:52061)
0619 14:57:33.981440 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:33.981457 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:33.981603 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:33.981978 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:33.981982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:33.982393 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:33.982560 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:33.982569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.983677 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.983681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:33.984596 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:33.984607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.985423 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.985428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:33.985599 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:33.986279 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:33.986305 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:33.989802 (+ 3497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":311,"threads_started":1}
W20260619 14:57:33.990329 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:33.981022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:52061 (local address 127.11.175.254:38869)
0619 14:57:33.981329 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:33.981335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:33.981555 (+ 220us) server_negotiation.cc:407] Connection header received
0619 14:57:33.981719 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:33.981723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:33.981786 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:33.981875 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:33.982765 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.983547 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.984744 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:33.985236 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:33.985365 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:33.990037 (+ 4672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:52061: BlockingRecv error: recv got EOF from 127.11.175.193:52061 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Jun 19 14:57:34 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:34 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:34.468252 21248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:34.459460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:50883)
0619 14:57:34.460038 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:34.460058 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:34.460175 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:34.460467 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:34.460470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:34.460896 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:34.461058 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:34.461066 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.462072 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.462076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:34.463343 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:34.463356 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.464265 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.464270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:34.464385 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:34.464949 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:34.464967 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:34.467895 (+ 2928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":282,"threads_started":1}
W20260619 14:57:34.468294 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:34.459563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:50883 (local address 127.11.175.254:38869)
0619 14:57:34.459774 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:34.459778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:34.460130 (+ 352us) server_negotiation.cc:407] Connection header received
0619 14:57:34.460270 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:34.460273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:34.460334 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:34.460428 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:34.461219 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.461950 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.463553 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.464117 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.464234 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:34.468097 (+ 3863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:50883: BlockingRecv error: recv got EOF from 127.11.175.194:50883 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Jun 19 14:57:34 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:34 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:34.945245 21249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:34.932246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:50353)
0619 14:57:34.932800 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:34.932818 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:34.932939 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:34.933322 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:34.933325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:34.933735 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:34.933898 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:34.933907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.935205 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.935210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:34.936235 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:34.936248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.937074 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.937079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:34.937213 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:34.941874 (+ 4661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:34.941893 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:34.944988 (+ 3095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":270,"threads_started":1}
W20260619 14:57:34.945624 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:34.932410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:50353 (local address 127.11.175.254:38869)
0619 14:57:34.932681 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:34.932688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:34.932986 (+ 298us) server_negotiation.cc:407] Connection header received
0619 14:57:34.933063 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:34.933067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:34.933164 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:34.933273 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:34.934213 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.935042 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.936400 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.936835 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.936935 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:34.945263 (+ 8328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:50353: BlockingRecv error: recv got EOF from 127.11.175.195:50353 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Jun 19 14:57:34 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:34 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:35.000726 21250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:34.991073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:52071)
0619 14:57:34.991780 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:34.991799 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:34.991993 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:34.992391 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:34.992397 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:34.992942 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:34.993128 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:34.993139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.994349 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.994357 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:34.995554 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:34.995569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.996547 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.996555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:34.996755 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:34.997437 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:34.997459 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:35.000356 (+ 2897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":437,"threads_started":1}
W20260619 14:57:35.000826 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:34.991302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:52071 (local address 127.11.175.254:38869)
0619 14:57:34.991545 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:34.991553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:34.991898 (+ 345us) server_negotiation.cc:407] Connection header received
0619 14:57:34.992148 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:34.992163 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:34.992236 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:34.992369 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:34.993300 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.994174 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.995750 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:34.996311 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:34.996500 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:35.000574 (+ 4074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:52071: BlockingRecv error: recv got EOF from 127.11.175.193:52071 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 14:57:35 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:35 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:35.478890 21251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:35.469195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:36977)
0619 14:57:35.469813 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:35.469834 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:35.470051 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:35.470417 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:35.470422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:35.470874 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:35.471037 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:35.471046 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:35.472388 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:35.472394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:35.473680 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:35.473694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:35.474474 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:35.474482 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:35.474666 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:35.475284 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:35.475310 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:35.478597 (+ 3287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":339,"threads_started":1}
W20260619 14:57:35.479216 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:35.469330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:36977 (local address 127.11.175.254:38869)
0619 14:57:35.469610 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:35.469617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:35.469940 (+ 323us) server_negotiation.cc:407] Connection header received
0619 14:57:35.470198 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:35.470208 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:35.470274 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:35.470384 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:35.471217 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:35.472232 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:35.473835 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:35.474265 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:35.474400 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:35.478897 (+ 4497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:36977: BlockingRecv error: recv got EOF from 127.11.175.194:36977 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Jun 19 14:57:35 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:35 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:35.955910 21252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:35.946232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:41251)
0619 14:57:35.946865 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:35.946882 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:35.947062 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:35.947457 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:35.947462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:35.947879 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:35.948057 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:35.948066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:35.949121 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:35.949128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:35.950024 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:35.950034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:35.950786 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:35.950792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:35.950954 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:35.952033 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:35.952066 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:35.955690 (+ 3624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":415,"threads_started":1}
W20260619 14:57:35.956283 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:35.946332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:41251 (local address 127.11.175.254:38869)
0619 14:57:35.946613 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:35.946622 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:35.946977 (+ 355us) server_negotiation.cc:407] Connection header received
0619 14:57:35.947231 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:35.947235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:35.947305 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:35.947419 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:35.948200 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:35.948980 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:35.950168 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:35.950581 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:35.950732 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:35.955936 (+ 5204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:41251: BlockingRecv error: recv got EOF from 127.11.175.195:41251 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Jun 19 14:57:36 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:36 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:36.011072 21253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:36.001854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:41511)
0619 14:57:36.002499 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:36.002520 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:36.002712 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:36.003067 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:36.003074 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:36.003481 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:36.003645 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:36.003653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.004673 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.004679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:36.005659 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:36.005671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.006456 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.006462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:36.006706 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:36.007367 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:36.007388 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:36.010573 (+ 3185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":313,"threads_started":1}
W20260619 14:57:36.011145 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:36.001931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:41511 (local address 127.11.175.254:38869)
0619 14:57:36.002152 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:36.002162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:36.002601 (+ 439us) server_negotiation.cc:407] Connection header received
0619 14:57:36.002834 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:36.002839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:36.002911 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:36.003021 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:36.003790 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.004523 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.005828 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.006257 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.006439 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:36.010845 (+ 4406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:41511: BlockingRecv error: recv got EOF from 127.11.175.193:41511 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 19 14:57:36 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:36 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:36.489142 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:36.479901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:36211)
0619 14:57:36.480442 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:36.480463 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:36.480575 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:36.480910 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:36.480913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:36.481284 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:36.481444 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:36.481453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.482443 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.482447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:36.483503 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:36.483516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.484265 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.484271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:36.484434 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:36.485180 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:36.485198 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:36.488762 (+ 3564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":297,"threads_started":1}
W20260619 14:57:36.489212 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:36.479997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:36211 (local address 127.11.175.254:38869)
0619 14:57:36.480162 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:36.480168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:36.480533 (+ 365us) server_negotiation.cc:407] Connection header received
0619 14:57:36.480697 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:36.480701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:36.480762 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:36.480843 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:36.481569 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.482307 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.483631 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.484107 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.484218 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:36.488974 (+ 4756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:36211: BlockingRecv error: recv got EOF from 127.11.175.194:36211 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 14:57:36 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:36 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:36.965862 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:36.956919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:54617)
0619 14:57:36.957488 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:36.957504 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:36.957622 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:36.957980 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:36.957984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:36.958437 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:36.958607 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:36.958616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.959688 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.959693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:36.960727 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:36.960740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.961533 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.961537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:36.961665 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:36.962367 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:36.962386 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:36.965615 (+ 3229us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":313,"threads_started":1}
W20260619 14:57:36.966282 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:36.957026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:54617 (local address 127.11.175.254:38869)
0619 14:57:36.957282 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:36.957290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:36.957581 (+ 291us) server_negotiation.cc:407] Connection header received
0619 14:57:36.957726 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:36.957730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:36.957806 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:36.957898 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:36.958785 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.959544 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.960876 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:36.961376 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:36.961488 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:36.965861 (+ 4373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:54617: BlockingRecv error: recv got EOF from 127.11.175.195:54617 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
Jun 19 14:57:37 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:37 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:37.021763 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:37.012269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:50769)
0619 14:57:37.012864 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:37.012883 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:37.013050 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:37.013475 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:37.013479 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:37.013868 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:37.014032 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:37.014041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.015171 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.015176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:37.016206 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:37.016221 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.016982 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.016986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:37.017099 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:37.017817 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:37.017835 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:37.021433 (+ 3598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":338,"threads_started":1}
W20260619 14:57:37.021930 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:37.012525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:50769 (local address 127.11.175.254:38869)
0619 14:57:37.012784 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:37.012790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:37.013001 (+ 211us) server_negotiation.cc:407] Connection header received
0619 14:57:37.013193 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:37.013197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:37.013263 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:37.013358 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:37.014208 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.014992 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.016380 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.016830 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.016942 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:37.021698 (+ 4756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:50769: BlockingRecv error: recv got EOF from 127.11.175.193:50769 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Jun 19 14:57:37 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:37 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:37.499193 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:37.490148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:42429)
0619 14:57:37.490651 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:37.490673 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:37.490785 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:37.491112 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:37.491116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:37.491581 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:37.491820 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:37.491837 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.493247 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.493252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:37.494197 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:37.494208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.495122 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.495127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:37.495258 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:37.495886 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:37.495911 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:37.498931 (+ 3020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":262,"threads_started":1}
W20260619 14:57:37.499413 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:37.490284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:42429 (local address 127.11.175.254:38869)
0619 14:57:37.490514 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:37.490520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:37.490747 (+ 227us) server_negotiation.cc:407] Connection header received
0619 14:57:37.490884 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:37.490888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:37.490950 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:37.491061 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:37.491970 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.493066 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.494346 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.494906 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.495007 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:37.499137 (+ 4130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:42429: BlockingRecv error: recv got EOF from 127.11.175.194:42429 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Jun 19 14:57:37 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:37 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:37.975970 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:37.966912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:53813)
0619 14:57:37.967533 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:37.967552 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:37.967672 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:37.968080 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:37.968084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:37.968445 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:37.968592 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:37.968601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.969614 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.969618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:37.970590 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:37.970601 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.971479 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.971483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:37.971674 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:37.972408 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:37.972427 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:37.975593 (+ 3166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":358,"threads_started":1}
W20260619 14:57:37.976085 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:37.967084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:53813 (local address 127.11.175.254:38869)
0619 14:57:37.967316 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:37.967321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:37.967629 (+ 308us) server_negotiation.cc:407] Connection header received
0619 14:57:37.967807 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:37.967811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:37.967869 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:37.967955 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:37.968750 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.969474 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.970770 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:37.971300 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:37.971407 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:37.975813 (+ 4406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:53813: BlockingRecv error: recv got EOF from 127.11.175.195:53813 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Jun 19 14:57:38 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:38 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:38.032455 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:38.022914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:39103)
0619 14:57:38.023514 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:38.023543 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:38.023678 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:38.024011 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:38.024014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:38.024428 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:38.024629 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:38.024638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.026219 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.026225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:38.027440 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:38.027453 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.028184 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.028188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:38.028307 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:38.028910 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:38.028927 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:38.032182 (+ 3255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":350,"threads_started":1}
W20260619 14:57:38.032701 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:38.023115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:39103 (local address 127.11.175.254:38869)
0619 14:57:38.023331 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:38.023338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:38.023665 (+ 327us) server_negotiation.cc:407] Connection header received
0619 14:57:38.023775 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:38.023780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:38.023845 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:38.023978 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:38.024863 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.026074 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.027599 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.028048 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.028175 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:38.032428 (+ 4253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:39103: BlockingRecv error: recv got EOF from 127.11.175.193:39103 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 14:57:38 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:38 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:38.509017 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:38.500241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:38657)
0619 14:57:38.501045 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:38.501068 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:38.501244 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:38.501622 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:38.501626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:38.502033 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:38.502205 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:38.502213 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.503314 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.503318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:38.504211 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:38.504222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.504968 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.504972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:38.505092 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:38.505739 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:38.505756 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:38.508774 (+ 3018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":542,"threads_started":1}
W20260619 14:57:38.509328 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:38.500344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:38657 (local address 127.11.175.254:38869)
0619 14:57:38.500623 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:38.500632 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:38.501256 (+ 624us) server_negotiation.cc:407] Connection header received
0619 14:57:38.501381 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:38.501386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:38.501451 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:38.501545 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:38.502377 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.503193 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.504361 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.504832 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.504937 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:38.509041 (+ 4104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:38657: BlockingRecv error: recv got EOF from 127.11.175.194:38657 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Jun 19 14:57:38 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:38 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:38.986320 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:38.977073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:53497)
0619 14:57:38.977698 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:38.977714 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:38.977887 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:38.978260 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:38.978265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:38.978764 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:38.978945 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:38.978954 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.980043 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.980050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:38.981010 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:38.981026 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.982162 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.982167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:38.982391 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:38.983204 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:38.983222 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:38.986068 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":341,"threads_started":1}
W20260619 14:57:38.986588 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:38.977337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:53497 (local address 127.11.175.254:38869)
0619 14:57:38.977597 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:38.977605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:38.977815 (+ 210us) server_negotiation.cc:407] Connection header received
0619 14:57:38.978033 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:38.978038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:38.978109 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:38.978224 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:38.979176 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.979902 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.981205 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:38.981971 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:38.982130 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:38.986315 (+ 4185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:53497: BlockingRecv error: recv got EOF from 127.11.175.195:53497 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Jun 19 14:57:39 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:39 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:39.042339 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:39.033583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:57323)
0619 14:57:39.034172 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:39.034195 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:39.034331 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:39.034762 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:39.034767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:39.035221 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:39.035375 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:39.035383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.036446 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.036450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:39.037404 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:39.037417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.038117 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.038121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:39.038291 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:39.038990 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:39.039008 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:39.041990 (+ 2982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":353,"threads_started":1}
W20260619 14:57:39.042474 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:39.033682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:57323 (local address 127.11.175.254:38869)
0619 14:57:39.033885 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:39.033890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:39.034309 (+ 419us) server_negotiation.cc:407] Connection header received
0619 14:57:39.034482 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:39.034486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:39.034557 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:39.034730 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:39.035554 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.036297 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.037549 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.037982 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.038079 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:39.042241 (+ 4162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:57323: BlockingRecv error: recv got EOF from 127.11.175.193:57323 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 19 14:57:39 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:39 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:39.519691 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:39.510258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:46959)
0619 14:57:39.510962 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:39.510993 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:39.511127 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:39.511501 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:39.511504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:39.511933 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:39.512167 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:39.512179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.513395 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.513399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:39.514614 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:39.514627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.515656 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.515663 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:39.515849 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:39.516528 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:39.516546 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:39.519432 (+ 2886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":424,"threads_started":1}
W20260619 14:57:39.520190 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:39.510362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:46959 (local address 127.11.175.254:38869)
0619 14:57:39.510588 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:39.510593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:39.511119 (+ 526us) server_negotiation.cc:407] Connection header received
0619 14:57:39.511228 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:39.511232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:39.511299 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:39.511395 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:39.512333 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.513168 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.514833 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.515468 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.515609 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:39.519643 (+ 4034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:46959: BlockingRecv error: recv got EOF from 127.11.175.194:46959 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 19 14:57:39 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:39 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:40.000036 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:39.987279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:53055)
0619 14:57:39.987812 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:39.987829 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:39.987941 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:39.988276 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:39.988279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:39.988751 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:39.988909 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:39.988918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.989916 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.989920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:39.990903 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:39.990915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.991598 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.991602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:39.991731 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:39.996581 (+ 4850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:39.996601 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:39.999806 (+ 3205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":331,"threads_started":1}
W20260619 14:57:40.000216 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:39.987373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:53055 (local address 127.11.175.254:38869)
0619 14:57:39.987789 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:39.987794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:39.987907 (+ 113us) server_negotiation.cc:407] Connection header received
0619 14:57:39.988052 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:39.988056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:39.988125 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:39.988231 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:39.989024 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.989781 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.991029 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:39.991474 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:39.991568 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:40.000004 (+ 8436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:53055: BlockingRecv error: recv got EOF from 127.11.175.195:53055 (error 108)
Metrics: {"server-negotiator.queue_time_us":231}
Jun 19 14:57:40 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:40 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:40.053825 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:40.043622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:40847)
0619 14:57:40.044211 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:40.044227 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:40.044344 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:40.044750 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:40.044753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:40.045142 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:40.045336 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:40.045345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:40.046492 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:40.046496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:40.047533 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:40.047549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:40.048708 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:40.048719 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:40.048928 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:40.049953 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:40.049978 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:40.053545 (+ 3567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":297,"threads_started":1}
W20260619 14:57:40.054316 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:40.043713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:40847 (local address 127.11.175.254:38869)
0619 14:57:40.043943 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:40.043948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:40.044335 (+ 387us) server_negotiation.cc:407] Connection header received
0619 14:57:40.044514 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:40.044518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:40.044596 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:40.044689 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:40.045490 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:40.046312 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:40.047731 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:40.048457 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:40.048605 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:40.054002 (+ 5397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:40847: BlockingRecv error: recv got EOF from 127.11.175.193:40847 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Jun 19 14:57:40 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:40 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:40.529186 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:40.520684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:55045)
0619 14:57:40.521239 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:40.521258 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:40.521371 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:40.521676 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:40.521679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:40.522056 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:40.522243 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:40.522252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:40.523504 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:40.523508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:40.524479 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:40.524492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:40.525244 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:40.525248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:40.525363 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:40.526003 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:40.526021 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:40.528950 (+ 2929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":286,"threads_started":1}
W20260619 14:57:40.529537 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:40.520784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:55045 (local address 127.11.175.254:38869)
0619 14:57:40.520959 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:40.520964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:40.521337 (+ 373us) server_negotiation.cc:407] Connection header received
0619 14:57:40.521463 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:40.521466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:40.521540 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:40.521622 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:40.522395 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:40.523381 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:40.524612 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:40.525062 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:40.525156 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:40.529208 (+ 4052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:55045: BlockingRecv error: recv got EOF from 127.11.175.194:55045 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 14:57:41 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:41 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:41.009109 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:41.001004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:40549)
0619 14:57:41.001549 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:41.001565 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:41.001679 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:41.001960 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:41.001963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:41.002322 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:41.002471 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:41.002479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.003423 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.003426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:41.004344 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:41.004354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.005084 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.005088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:41.005200 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:41.005785 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:41.005802 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:41.008770 (+ 2968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":292,"threads_started":1}
W20260619 14:57:41.009290 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:41.001125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:40549 (local address 127.11.175.254:38869)
0619 14:57:41.001314 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:41.001318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:41.001643 (+ 325us) server_negotiation.cc:407] Connection header received
0619 14:57:41.001775 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:41.001778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:41.001838 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:41.001930 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:41.002588 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.003305 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.004494 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.004935 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.005034 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:41.008999 (+ 3965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:40549: BlockingRecv error: recv got EOF from 127.11.175.195:40549 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 14:57:41 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:41 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:41.063803 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:41.055051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:44015)
0619 14:57:41.055555 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:41.055571 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:41.055689 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:41.055995 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:41.055998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:41.056375 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:41.056573 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:41.056583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.057866 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.057871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:41.059203 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:41.059217 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.059850 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.059854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:41.059971 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:41.060719 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:41.060737 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:41.063541 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":258,"threads_started":1}
W20260619 14:57:41.063956 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:41.055177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:44015 (local address 127.11.175.254:38869)
0619 14:57:41.055545 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:41.055551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:41.055646 (+ 95us) server_negotiation.cc:407] Connection header received
0619 14:57:41.055803 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:41.055807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:41.055867 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:41.055948 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:41.056707 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.057683 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.059331 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.059719 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.059810 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:41.063722 (+ 3912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:44015: BlockingRecv error: recv got EOF from 127.11.175.193:44015 (error 108)
Metrics: {"server-negotiator.queue_time_us":220}
Jun 19 14:57:41 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:41 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:41.538683 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:41.530187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:50395)
0619 14:57:41.530713 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:41.530738 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:41.530860 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:41.531160 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:41.531163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:41.531524 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:41.531692 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:41.531700 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.532674 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.532677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:41.533619 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:41.533630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.534447 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.534453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:41.534655 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:41.535351 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:41.535371 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:41.538393 (+ 3022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":296,"threads_started":1}
W20260619 14:57:41.538918 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:41.530290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:50395 (local address 127.11.175.254:38869)
0619 14:57:41.530484 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:41.530489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:41.530823 (+ 334us) server_negotiation.cc:407] Connection header received
0619 14:57:41.530959 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:41.530963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:41.531024 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:41.531112 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:41.531813 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.532556 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.533738 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:41.534280 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:41.534404 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:41.538617 (+ 4213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:50395: BlockingRecv error: recv got EOF from 127.11.175.194:50395 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Jun 19 14:57:42 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:42 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.195@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:42.019217 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:42.010039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.195:41779)
0619 14:57:42.010527 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:42.010544 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:42.010752 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:42.011134 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:42.011138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:42.011609 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:42.011801 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:42.011810 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.012842 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.012846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:42.013797 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:42.013809 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.014493 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.014497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:42.014614 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:42.015392 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:42.015411 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:42.018875 (+ 3464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":234,"threads_started":1}
W20260619 14:57:42.019320 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:42.010195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:41779 (local address 127.11.175.254:38869)
0619 14:57:42.010428 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:42.010434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:42.010676 (+ 242us) server_negotiation.cc:407] Connection header received
0619 14:57:42.010864 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:42.010868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:42.010957 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:42.011041 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:42.011941 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.012700 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.013935 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.014367 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.014462 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:42.019091 (+ 4629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.195:41779: BlockingRecv error: recv got EOF from 127.11.175.195:41779 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Jun 19 14:57:42 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:42 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.193@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:42.074013 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:42.065314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.193:45423)
0619 14:57:42.065908 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:42.065924 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:42.066090 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:42.066460 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:42.066464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:42.066938 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:42.067107 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:42.067116 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.068129 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.068135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:42.069069 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:42.069082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.069792 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.069798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:42.069979 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:42.070653 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:42.070674 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:42.073658 (+ 2984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":340,"threads_started":1}
W20260619 14:57:42.074177 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:42.065414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:45423 (local address 127.11.175.254:38869)
0619 14:57:42.065625 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:42.065631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:42.066024 (+ 393us) server_negotiation.cc:407] Connection header received
0619 14:57:42.066226 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:42.066232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:42.066304 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:42.066409 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:42.067271 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.067981 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.069221 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.069634 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.069775 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:42.073892 (+ 4117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.193:45423: BlockingRecv error: recv got EOF from 127.11.175.193:45423 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 19 14:57:42 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 14:57:42 dist-test-slave-76rk krb5kdc[20225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.175.194@KRBTEST.COM for oryx/127.11.175.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 14:57:42.548496 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:42.539866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38869 (local address 127.11.175.194:37989)
0619 14:57:42.540396 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:42.540422 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:42.540598 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:42.540958 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:42.540965 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:42.541400 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:42.541579 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:42.541587 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.542718 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.542723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:42.543663 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:42.543674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.544397 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.544402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:42.544575 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:42.545211 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:42.545229 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:42.548080 (+ 2851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.175.254:38869: Server oryx/127.11.175.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":377,"threads_started":1}
W20260619 14:57:42.548568 21227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:42.539983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:37989 (local address 127.11.175.254:38869)
0619 14:57:42.540231 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:42.540238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:42.540523 (+ 285us) server_negotiation.cc:407] Connection header received
0619 14:57:42.540725 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:42.540730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:42.540807 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:42.540915 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:42.541724 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.542535 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.543809 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.544247 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.544385 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:42.548300 (+ 3915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.175.194:37989: BlockingRecv error: recv got EOF from 127.11.175.194:37989 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 14:57:42.840585 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 20814
I20260619 14:57:42.853647 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:42.841860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52730 (local address 127.11.175.193:37811)
0619 14:57:42.842447 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:42.842455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:42.842483 (+ 28us) server_negotiation.cc:407] Connection header received
0619 14:57:42.842543 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:42.842549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:42.842859 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:42.843013 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:42.844434 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.845475 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.846703 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.847237 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.847399 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:42.850159 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:42.850167 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:42.850182 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:42.850228 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:42.852984 (+ 2756us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.175.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 14:57:42.853365 (+ 381us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52730: Request ticket server kudu/127.11.175.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":304,"threads_started":1}
W20260619 14:57:42.853760 21274 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52730: Request ticket server kudu/127.11.175.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 14:57:42.853820 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 20814: Not authorized: Client connection negotiation failed: client connection to 127.11.175.193:37811: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.175.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 14:57:42.854079 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 20814
I20260619 14:57:42.874425 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 20965
I20260619 14:57:42.887466 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:42.875708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57066 (local address 127.11.175.194:39739)
0619 14:57:42.876432 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:42.876439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:42.876469 (+ 30us) server_negotiation.cc:407] Connection header received
0619 14:57:42.876529 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:42.876536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:42.876774 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:42.876927 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:42.878253 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.879437 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.880612 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.881246 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.881346 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:42.883974 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:42.883982 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:42.883996 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:42.884031 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:42.886914 (+ 2883us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.175.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 14:57:42.887186 (+ 272us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57066: Request ticket server kudu/127.11.175.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":365,"threads_started":1}
W20260619 14:57:42.887585 21275 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57066: Request ticket server kudu/127.11.175.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 14:57:42.887650 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 20965: Not authorized: Client connection negotiation failed: client connection to 127.11.175.194:39739: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.175.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 14:57:42.887847 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 20965
I20260619 14:57:42.904433 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 21096
I20260619 14:57:42.918144 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:42.905529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.11.175.195:42877)
0619 14:57:42.906189 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:42.906196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:42.906224 (+ 28us) server_negotiation.cc:407] Connection header received
0619 14:57:42.906287 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:42.906294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:42.906515 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:42.906789 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:42.908155 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.909254 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.910906 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:42.911571 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:42.911694 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:42.914702 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:42.914722 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:42.914737 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:42.914779 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:42.917590 (+ 2811us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.175.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 14:57:42.917892 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59520: Request ticket server kudu/127.11.175.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":347,"threads_started":1}
W20260619 14:57:42.918298 21276 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59520: Request ticket server kudu/127.11.175.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 14:57:42.918500 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 21096: Not authorized: Client connection negotiation failed: client connection to 127.11.175.195:42877: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.175.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 14:57:42.918896 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 21096
I20260619 14:57:42.935529 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 20745
I20260619 14:57:42.937001 20807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:43.055680 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 20745
2026-06-19T14:57:43Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21410 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T14:57:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:57:43Z Disabled control of system clock
I20260619 14:57:43.137884 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:35811
--webserver_interface=127.11.175.254
--webserver_port=0
--builtin_ntp_servers=127.11.175.212:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:35811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 14:57:43.504640 21281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:43.504925 21281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:43.505017 21281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:43.512311 21281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:43.512424 21281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:43.512463 21281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:43.512513 21281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:43.512552 21281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:43.521862 21281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:35811
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:35811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:43.523221 21281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:43.524933 21281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:43.536736 21289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:43.536760 21287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:43.536877 21286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:43.540231 21281 server_base.cc:1034] running on GCE node
I20260619 14:57:43.541026 21281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:43.542447 21281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:43.543676 21281 hybrid_clock.cc:648] HybridClock initialized: now 1781881063543649 us; error 41 us; skew 500 ppm
I20260619 14:57:43.544027 21281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:57:43.546303 21281 webserver.cc:460] Webserver started at http://127.11.175.254:43617/ using document root <none> and password file <none>
I20260619 14:57:43.547098 21281 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:43.547240 21281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:43.547569 21281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:43.550591 21281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "575c91919d4e41d7a0e45ee090bf5c46"
format_stamp: "Formatted at 2026-06-19 14:57:43 on dist-test-slave-76rk"
server_key: "7ee46fd08c10a0ec1f33d62d7f4aa2db"
server_key_iv: "17d9c34473616cbb3120553f5dea5fa7"
server_key_version: "encryptionkey@0"
I20260619 14:57:43.551582 21281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "575c91919d4e41d7a0e45ee090bf5c46"
format_stamp: "Formatted at 2026-06-19 14:57:43 on dist-test-slave-76rk"
server_key: "7ee46fd08c10a0ec1f33d62d7f4aa2db"
server_key_iv: "17d9c34473616cbb3120553f5dea5fa7"
server_key_version: "encryptionkey@0"
I20260619 14:57:43.556928 21281 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:57:43.560550 21295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:43.562214 21281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 14:57:43.562415 21281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "575c91919d4e41d7a0e45ee090bf5c46"
format_stamp: "Formatted at 2026-06-19 14:57:43 on dist-test-slave-76rk"
server_key: "7ee46fd08c10a0ec1f33d62d7f4aa2db"
server_key_iv: "17d9c34473616cbb3120553f5dea5fa7"
server_key_version: "encryptionkey@0"
I20260619 14:57:43.562690 21281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:43.577775 21281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:43.578576 21281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:43.578896 21281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:43.598421 21281 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:35811
I20260619 14:57:43.598444 21346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:35811 every 8 connection(s)
I20260619 14:57:43.599824 21281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:57:43.602035 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 21281
I20260619 14:57:43.602232 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:57:43.602967 11967 external_mini_cluster.cc:1428] Setting key 54ce45faa63a8ac63519fc07556088f1
I20260619 14:57:43.604067 21347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:43.618880 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:43.604966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51614 (local address 127.11.175.254:35811)
0619 14:57:43.606044 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:43.606059 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:43.606110 (+ 51us) server_negotiation.cc:407] Connection header received
0619 14:57:43.607879 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:43.607907 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:43.608536 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:43.608968 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:43.609511 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:43.611633 (+ 2122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:43.613014 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:43.614076 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:43.614271 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:43.615539 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:43.615551 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:43.615570 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:43.616386 (+ 816us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:43.616488 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:43.616520 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:43.616663 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:43.617130 (+ 467us) server_negotiation.cc:299] Negotiation successful
0619 14:57:43.617532 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":325,"threads_started":1}
I20260619 14:57:43.619544 21347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46: Bootstrap starting.
I20260619 14:57:43.622682 21347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:43.623698 21347 log.cc:824] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:43.626303 21347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46: No bootstrap required, opened a new log
I20260619 14:57:43.633621 21347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575c91919d4e41d7a0e45ee090bf5c46" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35811 } }
I20260619 14:57:43.634137 21347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:43.634280 21347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575c91919d4e41d7a0e45ee090bf5c46, State: Initialized, Role: FOLLOWER
I20260619 14:57:43.634966 21347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575c91919d4e41d7a0e45ee090bf5c46" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35811 } }
I20260619 14:57:43.635175 21347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:57:43.635277 21347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:57:43.635396 21347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:43.637444 21347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575c91919d4e41d7a0e45ee090bf5c46" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35811 } }
I20260619 14:57:43.637885 21347 leader_election.cc:302] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 575c91919d4e41d7a0e45ee090bf5c46; no voters:
I20260619 14:57:43.638494 21347 leader_election.cc:288] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:57:43.638773 21352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:43.639616 21352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [term 1 LEADER]: Becoming Leader. State: Replica: 575c91919d4e41d7a0e45ee090bf5c46, State: Running, Role: LEADER
I20260619 14:57:43.640188 21352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575c91919d4e41d7a0e45ee090bf5c46" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35811 } }
I20260619 14:57:43.640973 21347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:57:43.643491 21353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "575c91919d4e41d7a0e45ee090bf5c46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575c91919d4e41d7a0e45ee090bf5c46" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35811 } } }
I20260619 14:57:43.643956 21353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:43.643826 21354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 575c91919d4e41d7a0e45ee090bf5c46. Latest consensus state: current_term: 1 leader_uuid: "575c91919d4e41d7a0e45ee090bf5c46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575c91919d4e41d7a0e45ee090bf5c46" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35811 } } }
I20260619 14:57:43.644170 21354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:43.647048 21361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:57:43.652869 21361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:57:43.662004 21361 catalog_manager.cc:1347] Generated new cluster ID: 5433197f3e7044c2ae4e994c670abebc
I20260619 14:57:43.662132 21361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:57:43.691937 21361 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:57:43.693307 21361 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:57:43.702800 21361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 575c91919d4e41d7a0e45ee090bf5c46: Generated new TSK 0
I20260619 14:57:43.703913 21361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 14:57:43.717289 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35811
--builtin_ntp_servers=127.11.175.212:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 14:57:44.054930 21371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:44.055220 21371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:44.055387 21371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:44.062988 21371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:44.063128 21371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:44.063263 21371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:57:44.072561 21371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:44.073875 21371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:44.075663 21371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:44.088871 21377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:44.088826 21376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:44.088920 21379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:44.092105 21371 server_base.cc:1034] running on GCE node
I20260619 14:57:44.092801 21371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:44.093801 21371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:44.095033 21371 hybrid_clock.cc:648] HybridClock initialized: now 1781881064094990 us; error 57 us; skew 500 ppm
I20260619 14:57:44.095396 21371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:57:44.097711 21371 webserver.cc:460] Webserver started at http://127.11.175.193:36171/ using document root <none> and password file <none>
I20260619 14:57:44.098428 21371 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:44.098559 21371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:44.098929 21371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:44.101792 21371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "a9a2f1de2ccb41d3a7db7f1fa4310042"
format_stamp: "Formatted at 2026-06-19 14:57:44 on dist-test-slave-76rk"
server_key: "0a54138669b21d6d2ad8f042ab1b977c"
server_key_iv: "3af0e683b51889abb44ee7559a502fb0"
server_key_version: "encryptionkey@0"
I20260619 14:57:44.102589 21371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "a9a2f1de2ccb41d3a7db7f1fa4310042"
format_stamp: "Formatted at 2026-06-19 14:57:44 on dist-test-slave-76rk"
server_key: "0a54138669b21d6d2ad8f042ab1b977c"
server_key_iv: "3af0e683b51889abb44ee7559a502fb0"
server_key_version: "encryptionkey@0"
I20260619 14:57:44.107776 21371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260619 14:57:44.111119 21385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:44.112450 21371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 14:57:44.112627 21371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "a9a2f1de2ccb41d3a7db7f1fa4310042"
format_stamp: "Formatted at 2026-06-19 14:57:44 on dist-test-slave-76rk"
server_key: "0a54138669b21d6d2ad8f042ab1b977c"
server_key_iv: "3af0e683b51889abb44ee7559a502fb0"
server_key_version: "encryptionkey@0"
I20260619 14:57:44.112860 21371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:44.124369 21371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:44.125248 21371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:44.125559 21371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:44.126533 21371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:44.127975 21371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:44.128098 21371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:44.128238 21371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:44.128326 21371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:44.162882 21371 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:34375
I20260619 14:57:44.162932 21497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:34375 every 8 connection(s)
I20260619 14:57:44.164256 21371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:57:44.164776 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 21371
I20260619 14:57:44.164966 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:57:44.165843 11967 external_mini_cluster.cc:1428] Setting key 207e39ac4398374700f2da688131bd56
I20260619 14:57:44.169191 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35811
--builtin_ntp_servers=127.11.175.212:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 14:57:44.180006 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:44.167089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:40539 (local address 127.11.175.254:35811)
0619 14:57:44.167523 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:44.167530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:44.169198 (+ 1668us) server_negotiation.cc:407] Connection header received
0619 14:57:44.172025 (+ 2827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:44.172029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:44.172096 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:44.172184 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:44.173681 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:44.174422 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:44.176990 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:44.177411 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:44.177517 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:44.179255 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:44.179260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:44.179263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:44.179282 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:44.179325 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:44.179337 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:44.179409 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:44.179687 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 14:57:44.179767 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":205,"threads_started":1}
I20260619 14:57:44.181449 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:44.167869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35811 (local address 127.11.175.193:40539)
0619 14:57:44.169007 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:44.169061 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:44.171717 (+ 2656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:44.172455 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:44.172469 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:44.172602 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:44.173448 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:44.173483 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:44.174588 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:44.174605 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:44.176827 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:44.176845 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:44.177656 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:44.177664 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:44.178010 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:44.178885 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:57:44.178890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:44.178949 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:57:44.179051 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:44.179433 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:44.179446 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:44.179457 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 14:57:44.179644 (+ 187us) client_negotiation.cc:240] Negotiation successful
0619 14:57:44.180071 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":299,"threads_started":1}
I20260619 14:57:44.183138 21498 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35811
I20260619 14:57:44.183449 21498 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:44.184211 21498 heartbeater.cc:511] Master 127.11.175.254:35811 requested a full tablet report, sending...
I20260619 14:57:44.186430 21312 ts_manager.cc:194] Registered new tserver with Master: a9a2f1de2ccb41d3a7db7f1fa4310042 (127.11.175.193:34375)
I20260619 14:57:44.188606 21312 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.193:40539
W20260619 14:57:44.501513 21502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:44.501791 21502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:44.501873 21502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:44.509116 21502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:44.509231 21502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:44.509336 21502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:57:44.518688 21502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:44.519897 21502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:44.521642 21502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:44.535923 21510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:44.536134 21507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:44.536142 21508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:44.537356 21502 server_base.cc:1034] running on GCE node
I20260619 14:57:44.538180 21502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:44.539258 21502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:44.540457 21502 hybrid_clock.cc:648] HybridClock initialized: now 1781881064540413 us; error 27 us; skew 500 ppm
I20260619 14:57:44.540844 21502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:57:44.543390 21502 webserver.cc:460] Webserver started at http://127.11.175.194:35599/ using document root <none> and password file <none>
I20260619 14:57:44.544103 21502 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:44.544221 21502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:44.544559 21502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:44.547544 21502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "5c3c73f4657847e39b0a46dfc0ccdf37"
format_stamp: "Formatted at 2026-06-19 14:57:44 on dist-test-slave-76rk"
server_key: "f745013dd2fe59905a181426d40c0f9a"
server_key_iv: "1e2a1e38ecb0368e70f888f4511176d7"
server_key_version: "encryptionkey@0"
I20260619 14:57:44.548416 21502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "5c3c73f4657847e39b0a46dfc0ccdf37"
format_stamp: "Formatted at 2026-06-19 14:57:44 on dist-test-slave-76rk"
server_key: "f745013dd2fe59905a181426d40c0f9a"
server_key_iv: "1e2a1e38ecb0368e70f888f4511176d7"
server_key_version: "encryptionkey@0"
I20260619 14:57:44.553464 21502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 14:57:44.557169 21516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:44.558566 21502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 14:57:44.558768 21502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "5c3c73f4657847e39b0a46dfc0ccdf37"
format_stamp: "Formatted at 2026-06-19 14:57:44 on dist-test-slave-76rk"
server_key: "f745013dd2fe59905a181426d40c0f9a"
server_key_iv: "1e2a1e38ecb0368e70f888f4511176d7"
server_key_version: "encryptionkey@0"
I20260619 14:57:44.558954 21502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:44.586058 21502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:44.586999 21502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:44.587299 21502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:44.588413 21502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:44.589841 21502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:44.589928 21502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:44.590018 21502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:44.590080 21502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:44.626541 21502 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:43189
I20260619 14:57:44.626631 21628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:43189 every 8 connection(s)
I20260619 14:57:44.627877 21502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:57:44.628314 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 21502
I20260619 14:57:44.628490 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:57:44.629110 11967 external_mini_cluster.cc:1428] Setting key dd6f2b17f8d473ba70323e0cfe2625b0
I20260619 14:57:44.632522 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35811
--builtin_ntp_servers=127.11.175.212:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 14:57:44.640872 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:44.630768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:48389 (local address 127.11.175.254:35811)
0619 14:57:44.630963 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:44.630969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:44.632537 (+ 1568us) server_negotiation.cc:407] Connection header received
0619 14:57:44.634386 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:44.634392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:44.634482 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:44.634591 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:44.635768 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:44.636487 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:44.638085 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:44.638474 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:44.638615 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:44.640037 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:44.640042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:44.640045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:44.640064 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:44.640103 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:44.640116 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:44.640224 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:44.640451 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 14:57:44.640553 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 14:57:44.641976 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:44.631408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35811 (local address 127.11.175.194:48389)
0619 14:57:44.632386 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:44.632429 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:44.634117 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:44.634891 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:44.634900 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:44.634999 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:44.635574 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:44.635599 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:44.636649 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:44.636655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:44.637922 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:44.637934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:44.638743 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:44.638752 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:44.638982 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:44.639692 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:57:44.639695 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:44.639735 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:57:44.639817 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:44.640216 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:44.640228 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:44.640238 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 14:57:44.640434 (+ 196us) client_negotiation.cc:240] Negotiation successful
0619 14:57:44.640886 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":320,"threads_started":1}
I20260619 14:57:44.643446 21629 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35811
I20260619 14:57:44.643782 21629 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:44.644541 21629 heartbeater.cc:511] Master 127.11.175.254:35811 requested a full tablet report, sending...
I20260619 14:57:44.646067 21312 ts_manager.cc:194] Registered new tserver with Master: 5c3c73f4657847e39b0a46dfc0ccdf37 (127.11.175.194:43189)
I20260619 14:57:44.647353 21312 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.194:48389
W20260619 14:57:44.974588 21632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:44.974987 21632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:44.975095 21632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:44.983348 21632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:44.983474 21632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:44.983593 21632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:57:44.994395 21632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:44.995824 21632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:44.997491 21632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:45.010470 21637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:45.010555 21640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:45.010560 21638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:45.011234 21632 server_base.cc:1034] running on GCE node
I20260619 14:57:45.011876 21632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:45.012848 21632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:45.014146 21632 hybrid_clock.cc:648] HybridClock initialized: now 1781881065014105 us; error 51 us; skew 500 ppm
I20260619 14:57:45.014484 21632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:57:45.020994 21632 webserver.cc:460] Webserver started at http://127.11.175.195:33693/ using document root <none> and password file <none>
I20260619 14:57:45.021792 21632 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:45.021932 21632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:45.022275 21632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:45.025326 21632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "4ffdab3e2e584672b6691282cea35e48"
format_stamp: "Formatted at 2026-06-19 14:57:45 on dist-test-slave-76rk"
server_key: "b63078937291cce5578f60b786ef642a"
server_key_iv: "b3c6376fd48ab294f590bcf1717a009d"
server_key_version: "encryptionkey@0"
I20260619 14:57:45.026144 21632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "4ffdab3e2e584672b6691282cea35e48"
format_stamp: "Formatted at 2026-06-19 14:57:45 on dist-test-slave-76rk"
server_key: "b63078937291cce5578f60b786ef642a"
server_key_iv: "b3c6376fd48ab294f590bcf1717a009d"
server_key_version: "encryptionkey@0"
I20260619 14:57:45.031543 21632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260619 14:57:45.035019 21646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:45.036332 21632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 14:57:45.036533 21632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "4ffdab3e2e584672b6691282cea35e48"
format_stamp: "Formatted at 2026-06-19 14:57:45 on dist-test-slave-76rk"
server_key: "b63078937291cce5578f60b786ef642a"
server_key_iv: "b3c6376fd48ab294f590bcf1717a009d"
server_key_version: "encryptionkey@0"
I20260619 14:57:45.036769 21632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:45.051407 21632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:45.052250 21632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:45.052556 21632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:45.053468 21632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:45.055030 21632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:45.055173 21632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:45.055313 21632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:45.055408 21632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:45.089232 21632 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:35141
I20260619 14:57:45.089270 21758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:35141 every 8 connection(s)
I20260619 14:57:45.090591 21632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:57:45.097663 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 21632
I20260619 14:57:45.097831 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:57:45.098299 11967 external_mini_cluster.cc:1428] Setting key 9c1a52b958bbe6cf7da54a9dacc54e00
I20260619 14:57:45.103418 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:45.093342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:57127 (local address 127.11.175.254:35811)
0619 14:57:45.093534 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:45.093539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:45.094958 (+ 1419us) server_negotiation.cc:407] Connection header received
0619 14:57:45.096744 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:45.096749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:45.096843 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:45.096953 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:45.098112 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:45.098969 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:45.100783 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:45.101339 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:45.101433 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:45.102739 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:45.102744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:45.102747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:45.102765 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:45.102797 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:45.102809 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:45.102879 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:45.103077 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 14:57:45.103151 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 14:57:45.104439 21761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:45.093855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35811 (local address 127.11.175.195:57127)
0619 14:57:45.094807 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:45.094844 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:45.096467 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:45.097201 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:45.097210 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:45.097298 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:45.097945 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:45.097966 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:45.099128 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:45.099133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:45.100653 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:45.100668 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:45.101556 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:45.101562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:45.101762 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:45.102406 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:57:45.102408 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:45.102446 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:57:45.102519 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:45.102893 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:45.102900 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:45.102906 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 14:57:45.103055 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 14:57:45.103372 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":284,"threads_started":1}
I20260619 14:57:45.105774 21759 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35811
I20260619 14:57:45.106029 21759 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:45.106788 21759 heartbeater.cc:511] Master 127.11.175.254:35811 requested a full tablet report, sending...
I20260619 14:57:45.108057 21312 ts_manager.cc:194] Registered new tserver with Master: 4ffdab3e2e584672b6691282cea35e48 (127.11.175.195:35141)
I20260619 14:57:45.108826 21312 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.195:57127
I20260619 14:57:45.113454 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 14:57:45.119648 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:45.117983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51616 (local address 127.11.175.254:35811)
0619 14:57:45.118177 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:45.118182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:45.118712 (+ 530us) server_negotiation.cc:407] Connection header received
0619 14:57:45.118929 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:45.118936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:45.119028 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:45.119132 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:45.119482 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51616: BlockingRecv error: recv got EOF from 127.0.0.1:51616 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 14:57:45.120397 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 21371
I20260619 14:57:45.127841 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:45.121346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.11.175.193:34375)
0619 14:57:45.121843 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:45.121850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:45.121875 (+ 25us) server_negotiation.cc:407] Connection header received
0619 14:57:45.121930 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:45.121936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:45.122135 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:45.122282 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:45.122599 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:45.123641 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:45.124872 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:45.125427 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:45.125520 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:45.126512 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:45.126520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:45.126532 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:45.127174 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:45.127217 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:45.127237 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:45.127329 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:45.127451 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 14:57:45.127569 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":262,"threads_started":1}
I20260619 14:57:45.129009 21493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:45.210712 21498 heartbeater.cc:503] Master 127.11.175.254:35811 was elected leader, sending a full tablet report...
I20260619 14:57:45.214963 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 21371
I20260619 14:57:45.234839 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 21502
I20260619 14:57:45.243496 21770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:45.235774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.11.175.194:43189)
0619 14:57:45.236396 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:45.236403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:45.236430 (+ 27us) server_negotiation.cc:407] Connection header received
0619 14:57:45.236489 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:45.236495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:45.236723 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:45.236852 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:45.237254 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:45.238275 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:45.239590 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:45.240236 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:45.240332 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:45.241737 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:45.241745 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:45.241758 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:45.242494 (+ 736us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:45.242551 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:45.242574 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:45.242736 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:45.242971 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 14:57:45.243153 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":336,"threads_started":1}
I20260619 14:57:45.244764 21624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:45.327687 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 21502
I20260619 14:57:45.343880 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 21632
I20260619 14:57:45.352021 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:45.344806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.11.175.195:35141)
0619 14:57:45.345455 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:45.345461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:45.345491 (+ 30us) server_negotiation.cc:407] Connection header received
0619 14:57:45.345552 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:45.345559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:45.345771 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:45.345902 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:45.346303 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:45.347458 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:45.348731 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:45.349287 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:45.349386 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:45.350468 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:45.350476 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:45.350489 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:45.351149 (+ 660us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:45.351225 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:45.351251 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:45.351341 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:45.351482 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 14:57:45.351638 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":377,"threads_started":1}
I20260619 14:57:45.353267 21754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:45.436647 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 21632
I20260619 14:57:45.452831 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 21281
I20260619 14:57:45.453955 21342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:45.558688 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 21281
2026-06-19T14:57:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2496 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T14:57:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:57:45Z Disabled control of system clock
I20260619 14:57:45.635967 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41427
--webserver_interface=127.11.175.254
--webserver_port=0
--builtin_ntp_servers=127.11.175.212:37863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:41427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 14:57:45.978677 21778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:45.978969 21778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:45.979156 21778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:45.986698 21778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:45.986822 21778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:45.986925 21778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:45.987005 21778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:45.987097 21778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:45.996707 21778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:37863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:41427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:45.998039 21778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:45.999929 21778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:46.012295 21784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:46.012352 21786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:46.012394 21783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:46.013731 21778 server_base.cc:1034] running on GCE node
I20260619 14:57:46.014554 21778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:46.016086 21778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:46.017341 21778 hybrid_clock.cc:648] HybridClock initialized: now 1781881066017292 us; error 45 us; skew 500 ppm
I20260619 14:57:46.017717 21778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:57:46.020148 21778 webserver.cc:460] Webserver started at http://127.11.175.254:43563/ using document root <none> and password file <none>
I20260619 14:57:46.020862 21778 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:46.020996 21778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:46.021327 21778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:46.025456 21778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa"
format_stamp: "Formatted at 2026-06-19 14:57:46 on dist-test-slave-76rk"
server_key: "b5c623ccd1c16b9dca111a0441872ff3"
server_key_iv: "b767e140ce9178bb9e4d5cc33f52a3b2"
server_key_version: "encryptionkey@0"
I20260619 14:57:46.026324 21778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa"
format_stamp: "Formatted at 2026-06-19 14:57:46 on dist-test-slave-76rk"
server_key: "b5c623ccd1c16b9dca111a0441872ff3"
server_key_iv: "b767e140ce9178bb9e4d5cc33f52a3b2"
server_key_version: "encryptionkey@0"
I20260619 14:57:46.031633 21778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 14:57:46.035050 21792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:46.036384 21778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 14:57:46.036569 21778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa"
format_stamp: "Formatted at 2026-06-19 14:57:46 on dist-test-slave-76rk"
server_key: "b5c623ccd1c16b9dca111a0441872ff3"
server_key_iv: "b767e140ce9178bb9e4d5cc33f52a3b2"
server_key_version: "encryptionkey@0"
I20260619 14:57:46.036798 21778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:46.042508 21778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:46.043306 21778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:46.043610 21778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:46.062300 21778 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:41427
I20260619 14:57:46.062333 21843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:41427 every 8 connection(s)
I20260619 14:57:46.063712 21778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:57:46.067508 21844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:46.070204 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 21778
I20260619 14:57:46.070415 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:57:46.071208 11967 external_mini_cluster.cc:1428] Setting key 9fec09e6fbeb41b7e03b302e6bad05d9
I20260619 14:57:46.079917 21847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:46.073488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39650 (local address 127.11.175.254:41427)
0619 14:57:46.074546 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:46.074557 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:46.074597 (+ 40us) server_negotiation.cc:407] Connection header received
0619 14:57:46.076322 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:46.076344 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:46.076778 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:46.077202 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:46.078292 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:46.078298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:46.078311 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:46.078327 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:46.078390 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:46.078405 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:46.078654 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:46.078819 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 14:57:46.079056 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":345,"threads_started":1}
I20260619 14:57:46.080224 21844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa: Bootstrap starting.
I20260619 14:57:46.083627 21844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:46.084542 21844 log.cc:824] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:46.087188 21844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa: No bootstrap required, opened a new log
I20260619 14:57:46.094871 21844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41427 } }
I20260619 14:57:46.095232 21844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:46.095314 21844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7f9e6919bf4dafa7cd08585a51d9aa, State: Initialized, Role: FOLLOWER
I20260619 14:57:46.095923 21844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41427 } }
I20260619 14:57:46.096134 21844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:57:46.096232 21844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:57:46.096354 21844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:46.098773 21844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41427 } }
I20260619 14:57:46.099228 21844 leader_election.cc:302] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d7f9e6919bf4dafa7cd08585a51d9aa; no voters:
I20260619 14:57:46.099802 21844 leader_election.cc:288] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:57:46.099895 21849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:46.100682 21849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [term 1 LEADER]: Becoming Leader. State: Replica: 6d7f9e6919bf4dafa7cd08585a51d9aa, State: Running, Role: LEADER
I20260619 14:57:46.101281 21849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41427 } }
I20260619 14:57:46.101886 21844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:57:46.104475 21851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7f9e6919bf4dafa7cd08585a51d9aa. Latest consensus state: current_term: 1 leader_uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41427 } } }
I20260619 14:57:46.104974 21851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:46.104558 21850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7f9e6919bf4dafa7cd08585a51d9aa" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41427 } } }
I20260619 14:57:46.105158 21850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:46.105974 21856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:57:46.115326 21856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:57:46.124817 21856 catalog_manager.cc:1347] Generated new cluster ID: 1e881eddf20942f7921851e958aba829
I20260619 14:57:46.124959 21856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:57:46.149354 21856 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:57:46.149811 21856 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:57:46.164345 21856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d7f9e6919bf4dafa7cd08585a51d9aa: Generated new TSK 0
I20260619 14:57:46.165227 21856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 14:57:46.179119 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41427
--builtin_ntp_servers=127.11.175.212:37863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 14:57:46.518463 21868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:46.518842 21868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:46.519006 21868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:46.526757 21868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:46.526891 21868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:46.527099 21868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:57:46.536398 21868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:37863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:46.537709 21868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:46.539510 21868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:46.552456 21873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:46.552495 21876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:46.553052 21874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:46.556156 21868 server_base.cc:1034] running on GCE node
I20260619 14:57:46.556833 21868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:46.557845 21868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:46.559085 21868 hybrid_clock.cc:648] HybridClock initialized: now 1781881066559042 us; error 34 us; skew 500 ppm
I20260619 14:57:46.559430 21868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:57:46.561949 21868 webserver.cc:460] Webserver started at http://127.11.175.193:37165/ using document root <none> and password file <none>
I20260619 14:57:46.562759 21868 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:46.562901 21868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:46.563225 21868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:46.566133 21868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "2093e2321d8642f792e570ac121d6447"
format_stamp: "Formatted at 2026-06-19 14:57:46 on dist-test-slave-76rk"
server_key: "433df34f7e36b321c35801344d723a13"
server_key_iv: "3ad17fff5cf431490d2b0138a9b2c7fc"
server_key_version: "encryptionkey@0"
I20260619 14:57:46.566978 21868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "2093e2321d8642f792e570ac121d6447"
format_stamp: "Formatted at 2026-06-19 14:57:46 on dist-test-slave-76rk"
server_key: "433df34f7e36b321c35801344d723a13"
server_key_iv: "3ad17fff5cf431490d2b0138a9b2c7fc"
server_key_version: "encryptionkey@0"
I20260619 14:57:46.572151 21868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 14:57:46.575816 21882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:46.577210 21868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 14:57:46.577405 21868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "2093e2321d8642f792e570ac121d6447"
format_stamp: "Formatted at 2026-06-19 14:57:46 on dist-test-slave-76rk"
server_key: "433df34f7e36b321c35801344d723a13"
server_key_iv: "3ad17fff5cf431490d2b0138a9b2c7fc"
server_key_version: "encryptionkey@0"
I20260619 14:57:46.577654 21868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:46.583487 21868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:46.584293 21868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:46.584576 21868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:46.585539 21868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:46.587051 21868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:46.587169 21868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:46.587306 21868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:46.587393 21868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:46.620956 21868 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:43453
I20260619 14:57:46.620996 21994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:43453 every 8 connection(s)
I20260619 14:57:46.622325 21868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:57:46.624814 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 21868
I20260619 14:57:46.625031 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:57:46.625656 11967 external_mini_cluster.cc:1428] Setting key 6917d965541c990be9722b1e67581039
I20260619 14:57:46.629093 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41427
--builtin_ntp_servers=127.11.175.212:37863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 14:57:46.632556 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:46.626106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:56931 (local address 127.11.175.254:41427)
0619 14:57:46.626537 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:46.626544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:46.629989 (+ 3445us) server_negotiation.cc:407] Connection header received
0619 14:57:46.630060 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:46.630065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:46.630151 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:46.630279 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:46.631774 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:46.631780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:46.631785 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:46.631804 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:46.631857 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:46.631868 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:46.632003 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:46.632193 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 14:57:46.632267 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":685,"threads_started":1}
I20260619 14:57:46.633569 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:46.625654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41427 (local address 127.11.175.193:56931)
0619 14:57:46.626873 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:46.627229 (+ 356us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:46.629549 (+ 2320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:46.630585 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:46.630594 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:46.630749 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:46.631469 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:57:46.631472 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:46.631512 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:57:46.631600 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:46.631965 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:46.631975 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:46.631984 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 14:57:46.632283 (+ 299us) client_negotiation.cc:240] Negotiation successful
0619 14:57:46.632558 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":457,"threads_started":1}
I20260619 14:57:46.635113 21995 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41427
I20260619 14:57:46.635428 21995 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:46.636065 21995 heartbeater.cc:511] Master 127.11.175.254:41427 requested a full tablet report, sending...
I20260619 14:57:46.638151 21809 ts_manager.cc:194] Registered new tserver with Master: 2093e2321d8642f792e570ac121d6447 (127.11.175.193:43453)
W20260619 14:57:46.971843 21999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:46.972116 21999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:46.972299 21999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:46.979748 21999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:46.979858 21999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:46.980055 21999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:57:46.989779 21999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:37863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:46.991237 21999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:46.993108 21999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:47.005995 22007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:47.006071 22005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:47.006075 22004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:47.007112 21999 server_base.cc:1034] running on GCE node
I20260619 14:57:47.007754 21999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:47.008805 21999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:47.009999 21999 hybrid_clock.cc:648] HybridClock initialized: now 1781881067009930 us; error 62 us; skew 500 ppm
I20260619 14:57:47.010392 21999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:57:47.012908 21999 webserver.cc:460] Webserver started at http://127.11.175.194:38031/ using document root <none> and password file <none>
I20260619 14:57:47.013630 21999 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:47.013756 21999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:47.014091 21999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:47.017068 21999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "f737b2bc65fe464d9603a76f2b5db8d2"
format_stamp: "Formatted at 2026-06-19 14:57:47 on dist-test-slave-76rk"
server_key: "5ba000ae823c1bbccfa50ad3789a70c5"
server_key_iv: "6da1837f0fff0d3a494beb5ea69f8fb0"
server_key_version: "encryptionkey@0"
I20260619 14:57:47.017886 21999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "f737b2bc65fe464d9603a76f2b5db8d2"
format_stamp: "Formatted at 2026-06-19 14:57:47 on dist-test-slave-76rk"
server_key: "5ba000ae823c1bbccfa50ad3789a70c5"
server_key_iv: "6da1837f0fff0d3a494beb5ea69f8fb0"
server_key_version: "encryptionkey@0"
I20260619 14:57:47.023110 21999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260619 14:57:47.026504 22013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:47.027884 21999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:57:47.028072 21999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "f737b2bc65fe464d9603a76f2b5db8d2"
format_stamp: "Formatted at 2026-06-19 14:57:47 on dist-test-slave-76rk"
server_key: "5ba000ae823c1bbccfa50ad3789a70c5"
server_key_iv: "6da1837f0fff0d3a494beb5ea69f8fb0"
server_key_version: "encryptionkey@0"
I20260619 14:57:47.028309 21999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:47.033970 21999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:47.034835 21999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:47.035104 21999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:47.035991 21999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:47.037482 21999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:47.037609 21999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:47.037751 21999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:47.037840 21999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:47.077090 21999 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:40617
I20260619 14:57:47.077135 22125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:40617 every 8 connection(s)
I20260619 14:57:47.078543 21999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:57:47.086593 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:47.081116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:60869 (local address 127.11.175.254:41427)
0619 14:57:47.081284 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:47.081288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:47.082908 (+ 1620us) server_negotiation.cc:407] Connection header received
0619 14:57:47.084522 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:47.084527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:47.084579 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:47.084667 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:47.085946 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:47.085950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:47.085953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:47.085968 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:47.085999 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:47.086010 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:47.086103 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:47.086322 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 14:57:47.086371 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 14:57:47.087604 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:47.081695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41427 (local address 127.11.175.194:60869)
0619 14:57:47.082767 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:47.082803 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:47.084285 (+ 1482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:47.084871 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:47.084880 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:47.084974 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:47.085649 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:57:47.085651 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:47.085692 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:57:47.085775 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:47.086125 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:47.086134 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:47.086140 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 14:57:47.086377 (+ 237us) client_negotiation.cc:240] Negotiation successful
0619 14:57:47.086589 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":261,"threads_started":1}
I20260619 14:57:47.087915 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 21999
I20260619 14:57:47.088122 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:57:47.088575 11967 external_mini_cluster.cc:1428] Setting key 718a2a84a8163196e58f20f952b05aef
I20260619 14:57:47.089203 22126 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41427
I20260619 14:57:47.089546 22126 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:47.090147 22126 heartbeater.cc:511] Master 127.11.175.254:41427 requested a full tablet report, sending...
I20260619 14:57:47.091527 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41427
--builtin_ntp_servers=127.11.175.212:37863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 14:57:47.091733 21809 ts_manager.cc:194] Registered new tserver with Master: f737b2bc65fe464d9603a76f2b5db8d2 (127.11.175.194:40617)
W20260619 14:57:47.421795 22129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:47.422107 22129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:47.422201 22129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:47.429701 22129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:47.429811 22129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:47.429921 22129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:57:47.439364 22129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:37863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:47.440689 22129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:47.442519 22129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:47.455601 22135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:47.455704 22137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:47.455946 22134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:47.456772 22129 server_base.cc:1034] running on GCE node
I20260619 14:57:47.457432 22129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:47.458426 22129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:47.459656 22129 hybrid_clock.cc:648] HybridClock initialized: now 1781881067459608 us; error 44 us; skew 500 ppm
I20260619 14:57:47.459993 22129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 14:57:47.462328 22129 webserver.cc:460] Webserver started at http://127.11.175.195:44439/ using document root <none> and password file <none>
I20260619 14:57:47.463100 22129 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:47.463256 22129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:47.463591 22129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:47.466485 22129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "f75056bbd2f64236992e0e63cafcb5f8"
format_stamp: "Formatted at 2026-06-19 14:57:47 on dist-test-slave-76rk"
server_key: "07896be38ad83e91ba440483428cb782"
server_key_iv: "9d808e0d2e790aa28f2a1683ed0408c7"
server_key_version: "encryptionkey@0"
I20260619 14:57:47.467343 22129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "f75056bbd2f64236992e0e63cafcb5f8"
format_stamp: "Formatted at 2026-06-19 14:57:47 on dist-test-slave-76rk"
server_key: "07896be38ad83e91ba440483428cb782"
server_key_iv: "9d808e0d2e790aa28f2a1683ed0408c7"
server_key_version: "encryptionkey@0"
I20260619 14:57:47.472539 22129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:57:47.476039 22143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:47.477365 22129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:57:47.477559 22129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "f75056bbd2f64236992e0e63cafcb5f8"
format_stamp: "Formatted at 2026-06-19 14:57:47 on dist-test-slave-76rk"
server_key: "07896be38ad83e91ba440483428cb782"
server_key_iv: "9d808e0d2e790aa28f2a1683ed0408c7"
server_key_version: "encryptionkey@0"
I20260619 14:57:47.477793 22129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:47.483776 22129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:47.484575 22129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:47.484853 22129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:47.485766 22129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:47.487200 22129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:47.487317 22129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:47.487452 22129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:47.487540 22129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:47.520644 22129 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:33447
I20260619 14:57:47.520690 22255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:33447 every 8 connection(s)
I20260619 14:57:47.522007 22129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:57:47.524904 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 22129
I20260619 14:57:47.525115 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:57:47.525583 11967 external_mini_cluster.cc:1428] Setting key 2da341c9a0f214bb906e2ea968a69da8
I20260619 14:57:47.530306 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:47.524750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:54523 (local address 127.11.175.254:41427)
0619 14:57:47.524974 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:47.524983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:47.526590 (+ 1607us) server_negotiation.cc:407] Connection header received
0619 14:57:47.528376 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:47.528379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:47.528466 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:47.528577 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:47.529745 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:47.529749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:47.529752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:47.529766 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:47.529795 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:47.529806 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:47.529891 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:47.530048 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 14:57:47.530095 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 14:57:47.531243 22258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:47.525355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41427 (local address 127.11.175.195:54523)
0619 14:57:47.526425 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:47.526471 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:47.528090 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:47.528777 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:47.528785 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:47.528874 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:47.529458 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 14:57:47.529461 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:47.529497 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 14:57:47.529586 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:47.529887 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:47.529894 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:47.529899 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 14:57:47.530097 (+ 198us) client_negotiation.cc:240] Negotiation successful
0619 14:57:47.530283 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":356,"threads_started":1}
I20260619 14:57:47.532433 22256 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41427
I20260619 14:57:47.532750 22256 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:47.533332 22256 heartbeater.cc:511] Master 127.11.175.254:41427 requested a full tablet report, sending...
I20260619 14:57:47.534546 21809 ts_manager.cc:194] Registered new tserver with Master: f75056bbd2f64236992e0e63cafcb5f8 (127.11.175.195:33447)
I20260619 14:57:47.540757 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 14:57:47.547075 21998 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 14:57:47.545392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39656 (local address 127.11.175.254:41427)
0619 14:57:47.545551 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:47.545555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:47.546129 (+ 574us) server_negotiation.cc:407] Connection header received
0619 14:57:47.546283 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:47.546287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:47.546336 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:47.546419 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:47.546796 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39656: BlockingRecv error: recv got EOF from 127.0.0.1:39656 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 14:57:47.547755 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 21868
I20260619 14:57:47.551391 22265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:47.548813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.11.175.193:43453)
0619 14:57:47.549373 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:47.549380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:47.549404 (+ 24us) server_negotiation.cc:407] Connection header received
0619 14:57:47.549461 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:47.549466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:47.549672 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:47.549796 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:47.550687 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:47.550711 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:47.550724 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:47.550745 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:47.550820 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:47.550845 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:47.550961 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:47.551083 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 14:57:47.551161 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":281,"threads_started":1}
I20260619 14:57:47.552551 21990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:47.635838 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 21868
I20260619 14:57:47.650776 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 21999
I20260619 14:57:47.654605 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:47.651925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.11.175.194:40617)
0619 14:57:47.652511 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:47.652518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:47.652544 (+ 26us) server_negotiation.cc:407] Connection header received
0619 14:57:47.652604 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:47.652610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:47.652828 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:47.652957 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:47.653918 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:47.653924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:47.653936 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:47.653950 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:47.653995 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:47.654017 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:47.654125 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:47.654275 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 14:57:47.654362 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":320,"threads_started":1}
I20260619 14:57:47.655915 22121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:47.737082 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 21999
I20260619 14:57:47.753271 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 22129
I20260619 14:57:47.757364 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:47.754205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56686 (local address 127.11.175.195:33447)
0619 14:57:47.754909 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:47.754916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:47.754946 (+ 30us) server_negotiation.cc:407] Connection header received
0619 14:57:47.755011 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:47.755017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:47.755242 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:47.755417 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:47.756594 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:47.756600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:47.756613 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 14:57:47.756630 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:47.756670 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 14:57:47.756686 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:47.756876 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:47.756955 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 14:57:47.757062 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":319,"threads_started":1}
I20260619 14:57:47.758898 22251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:47.841939 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 22129
I20260619 14:57:47.858446 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 21778
I20260619 14:57:47.859766 21839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:47.973307 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 21778
2026-06-19T14:57:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2413 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:57:48 dist-test-slave-76rk krb5kdc[22274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:57:48 dist-test-slave-76rk krb5kdc[22274](info): set up 2 sockets
Jun 19 14:57:48 dist-test-slave-76rk krb5kdc[22274](info): commencing operation
krb5kdc: starting...
W20260619 14:57:49.083854 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:57:49 dist-test-slave-76rk krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:57:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:57:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:57:49.326802 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:36759
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:36759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:49.672993 22287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:49.673317 22287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:49.673417 22287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:49.680816 22287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:57:49.680951 22287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:49.681021 22287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:49.681087 22287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:57:49.681138 22287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:57:49.691191 22287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:36759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:36759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:49.692538 22287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:49.693918 22287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:49.706271 22292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:49.706364 22293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:49.706321 22295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:49.707180 22287 server_base.cc:1034] running on GCE node
I20260619 14:57:49.707919 22287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:49.709298 22287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:49.710512 22287 hybrid_clock.cc:648] HybridClock initialized: now 1781881069710442 us; error 52 us; skew 500 ppm
I20260619 14:57:49.710877 22287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:49 dist-test-slave-76rk krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881069, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:49.714030 22287 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:57:49.715612 22287 webserver.cc:460] Webserver started at http://127.11.175.254:39217/ using document root <none> and password file <none>
I20260619 14:57:49.716318 22287 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:49.716466 22287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:49.716785 22287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:49.719823 22287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "42327c647a7c45e0a608cd2ee5c1c343"
format_stamp: "Formatted at 2026-06-19 14:57:49 on dist-test-slave-76rk"
server_key: "fb1c33c902ebabea025c1e56f8e633e6"
server_key_iv: "69a71bb099e039253e7dc3f9e087822b"
server_key_version: "encryptionkey@0"
I20260619 14:57:49.720584 22287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "42327c647a7c45e0a608cd2ee5c1c343"
format_stamp: "Formatted at 2026-06-19 14:57:49 on dist-test-slave-76rk"
server_key: "fb1c33c902ebabea025c1e56f8e633e6"
server_key_iv: "69a71bb099e039253e7dc3f9e087822b"
server_key_version: "encryptionkey@0"
I20260619 14:57:49.725656 22287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 14:57:49.729218 22302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:49.730626 22287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 14:57:49.730891 22287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "42327c647a7c45e0a608cd2ee5c1c343"
format_stamp: "Formatted at 2026-06-19 14:57:49 on dist-test-slave-76rk"
server_key: "fb1c33c902ebabea025c1e56f8e633e6"
server_key_iv: "69a71bb099e039253e7dc3f9e087822b"
server_key_version: "encryptionkey@0"
I20260619 14:57:49.731096 22287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:49.747457 22287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:49.752017 22287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:49.752334 22287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:49.771967 22287 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:36759
I20260619 14:57:49.771999 22353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:36759 every 8 connection(s)
I20260619 14:57:49.773375 22287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:57:49.777761 22354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:49.781148 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 22287
I20260619 14:57:49.781400 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:57:49.782210 11967 external_mini_cluster.cc:1428] Setting key d13619e328c181c02876347cd2cc19cc
I20260619 14:57:49.791687 22354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343: Bootstrap starting.
I20260619 14:57:49.795791 22354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:49.797216 22354 log.cc:824] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343: Log is configured to *not* fsync() on all Append() calls
Jun 19 14:57:49 dist-test-slave-76rk krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:49.800537 22354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343: No bootstrap required, opened a new log
I20260619 14:57:49.807916 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:49.784458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54190 (local address 127.11.175.254:36759)
0619 14:57:49.785267 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:49.785279 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:49.785321 (+ 42us) server_negotiation.cc:407] Connection header received
0619 14:57:49.787041 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:49.787065 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:49.787476 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:49.787818 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:49.789818 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:49.791222 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:49.793177 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:49.793899 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:49.794042 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:49.799466 (+ 5424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:49.799475 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:49.799491 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:49.799529 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:49.803355 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:49.804147 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:49.804154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:49.804161 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:49.804345 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:49.804750 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:49.804756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:49.804759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:49.805339 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:49.805713 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:49.806165 (+ 452us) server_negotiation.cc:299] Negotiation successful
0619 14:57:49.806692 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":391,"threads_started":1}
I20260619 14:57:49.809290 22354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42327c647a7c45e0a608cd2ee5c1c343" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36759 } }
I20260619 14:57:49.809715 22354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:49.809830 22354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42327c647a7c45e0a608cd2ee5c1c343, State: Initialized, Role: FOLLOWER
I20260619 14:57:49.810482 22354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42327c647a7c45e0a608cd2ee5c1c343" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36759 } }
I20260619 14:57:49.810752 22354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:57:49.810909 22354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:57:49.811071 22354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:49.813329 22354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42327c647a7c45e0a608cd2ee5c1c343" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36759 } }
I20260619 14:57:49.813822 22354 leader_election.cc:302] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42327c647a7c45e0a608cd2ee5c1c343; no voters:
I20260619 14:57:49.814563 22354 leader_election.cc:288] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:57:49.814699 22359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:49.815845 22359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [term 1 LEADER]: Becoming Leader. State: Replica: 42327c647a7c45e0a608cd2ee5c1c343, State: Running, Role: LEADER
I20260619 14:57:49.816399 22359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42327c647a7c45e0a608cd2ee5c1c343" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36759 } }
I20260619 14:57:49.817031 22354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:57:49.819605 22360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42327c647a7c45e0a608cd2ee5c1c343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42327c647a7c45e0a608cd2ee5c1c343" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36759 } } }
I20260619 14:57:49.820045 22360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:49.821449 22361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42327c647a7c45e0a608cd2ee5c1c343. Latest consensus state: current_term: 1 leader_uuid: "42327c647a7c45e0a608cd2ee5c1c343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42327c647a7c45e0a608cd2ee5c1c343" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36759 } } }
I20260619 14:57:49.821769 22361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343 [sys.catalog]: This master's current role is: LEADER
I20260619 14:57:49.822839 22368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:57:49.827060 22368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:57:49.836143 22368 catalog_manager.cc:1347] Generated new cluster ID: 91081d83ebfc4678a6a87e3905d92a90
I20260619 14:57:49.836265 22368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:57:49.864809 22368 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:57:49.866386 22368 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:57:49.872967 22368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343: Generated new TSK 0
I20260619 14:57:49.873844 22368 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:57:49.934967 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36759
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:50.278347 22380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:50.278728 22380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:50.278900 22380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:50.286437 22380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:50.286599 22380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:50.286813 22380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:57:50.297341 22380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:50.298723 22380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:50.300083 22380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:50.313372 22385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:50.313448 22388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:50.313452 22386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:50.314529 22380 server_base.cc:1034] running on GCE node
I20260619 14:57:50.315163 22380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:50.316077 22380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:50.317284 22380 hybrid_clock.cc:648] HybridClock initialized: now 1781881070317218 us; error 58 us; skew 500 ppm
I20260619 14:57:50.317615 22380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:50 dist-test-slave-76rk krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881070, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:50.324798 22380 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:57:50.326422 22380 webserver.cc:460] Webserver started at http://127.11.175.193:37087/ using document root <none> and password file <none>
I20260619 14:57:50.327201 22380 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:50.327363 22380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:50.327683 22380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:50.330761 22380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "1e5035beba3342eb956aeea91a613760"
format_stamp: "Formatted at 2026-06-19 14:57:50 on dist-test-slave-76rk"
server_key: "35c995871b3d7ab19e8583e32c71ab91"
server_key_iv: "ba754725d33b8df640c9e32ae9519650"
server_key_version: "encryptionkey@0"
I20260619 14:57:50.331569 22380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "1e5035beba3342eb956aeea91a613760"
format_stamp: "Formatted at 2026-06-19 14:57:50 on dist-test-slave-76rk"
server_key: "35c995871b3d7ab19e8583e32c71ab91"
server_key_iv: "ba754725d33b8df640c9e32ae9519650"
server_key_version: "encryptionkey@0"
I20260619 14:57:50.337134 22380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:57:50.340909 22395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:50.342370 22380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 14:57:50.342557 22380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "1e5035beba3342eb956aeea91a613760"
format_stamp: "Formatted at 2026-06-19 14:57:50 on dist-test-slave-76rk"
server_key: "35c995871b3d7ab19e8583e32c71ab91"
server_key_iv: "ba754725d33b8df640c9e32ae9519650"
server_key_version: "encryptionkey@0"
I20260619 14:57:50.342846 22380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:50.370141 22380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:50.374786 22380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:50.375140 22380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:50.376143 22380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:50.377563 22380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:50.377635 22380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:50.377808 22380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:50.377868 22380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:50.411695 22380 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:33211
I20260619 14:57:50.411743 22507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:33211 every 8 connection(s)
I20260619 14:57:50.413038 22380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:57:50.413384 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 22380
I20260619 14:57:50.413620 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:57:50.414474 11967 external_mini_cluster.cc:1428] Setting key 1fe3bfad3117509bb4afa9c9065b81bb
Jun 19 14:57:50 dist-test-slave-76rk krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881070, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:50.435801 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:50.416012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:53989 (local address 127.11.175.254:36759)
0619 14:57:50.416523 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:50.416529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:50.417771 (+ 1242us) server_negotiation.cc:407] Connection header received
0619 14:57:50.419713 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:50.419722 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:50.419810 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:50.419927 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:50.421638 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:50.422827 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:50.424683 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:50.425075 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:50.425175 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:50.429445 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:50.429450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:50.429453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:50.429489 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:50.432218 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:50.433036 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:50.433041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:50.433044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:50.433166 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:50.433620 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:50.433627 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:50.433633 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:50.433923 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:50.434086 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:50.435039 (+ 953us) server_negotiation.cc:299] Negotiation successful
0619 14:57:50.435380 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":265,"threads_started":1}
I20260619 14:57:50.436780 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:50.416741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:36759 (local address 127.11.175.193:53989)
0619 14:57:50.417602 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:50.417646 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:50.419469 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:50.420173 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:50.420182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:50.420814 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:50.421465 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:50.421483 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:50.422968 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:50.422972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:50.424519 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:50.424536 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:50.425298 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:50.425307 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:50.425624 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:50.426372 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:50.426405 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:50.429155 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:50.432400 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:50.432410 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:50.432437 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:50.432886 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:50.433283 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:50.433288 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:50.433291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:50.433496 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:50.434059 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:50.434067 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:50.434564 (+ 497us) client_negotiation.cc:769] Sending connection context
0619 14:57:50.434913 (+ 349us) client_negotiation.cc:240] Negotiation successful
0619 14:57:50.435548 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":372,"threads_started":1}
I20260619 14:57:50.438488 22508 heartbeater.cc:347] Connected to a master server at 127.11.175.254:36759
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 14:57:50.438881 22508 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:50.439656 22508 heartbeater.cc:511] Master 127.11.175.254:36759 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
I20260619 14:57:50.441797 22319 ts_manager.cc:194] Registered new tserver with Master: 1e5035beba3342eb956aeea91a613760 (127.11.175.193:33211)
I20260619 14:57:50.443346 22319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:53989
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:57:50.469277 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36759
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:50.815279 22514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:50.815547 22514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:50.815737 22514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:50.823334 22514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:50.823441 22514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:50.823647 22514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:57:50.833462 22514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:50.834939 22514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:50.836346 22514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:50.849258 22522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:50.849336 22519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:50.849458 22520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:50.850517 22514 server_base.cc:1034] running on GCE node
I20260619 14:57:50.851156 22514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:50.852032 22514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:50.853226 22514 hybrid_clock.cc:648] HybridClock initialized: now 1781881070853184 us; error 36 us; skew 500 ppm
I20260619 14:57:50.853543 22514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:50 dist-test-slave-76rk krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881070, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:50.856709 22514 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:57:50.858058 22514 webserver.cc:460] Webserver started at http://127.11.175.194:43053/ using document root <none> and password file <none>
I20260619 14:57:50.858767 22514 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:50.858904 22514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:50.859177 22514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:50.862217 22514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "d8937c88921142f89bcf7c6f3862217f"
format_stamp: "Formatted at 2026-06-19 14:57:50 on dist-test-slave-76rk"
server_key: "ab1ea8f36d205960c8ccf55d0b6b0a93"
server_key_iv: "2714013aad5752ffa389983b0de2caf1"
server_key_version: "encryptionkey@0"
I20260619 14:57:50.863276 22514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "d8937c88921142f89bcf7c6f3862217f"
format_stamp: "Formatted at 2026-06-19 14:57:50 on dist-test-slave-76rk"
server_key: "ab1ea8f36d205960c8ccf55d0b6b0a93"
server_key_iv: "2714013aad5752ffa389983b0de2caf1"
server_key_version: "encryptionkey@0"
I20260619 14:57:50.870164 22514 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260619 14:57:50.873705 22529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:50.875131 22514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:57:50.875311 22514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "d8937c88921142f89bcf7c6f3862217f"
format_stamp: "Formatted at 2026-06-19 14:57:50 on dist-test-slave-76rk"
server_key: "ab1ea8f36d205960c8ccf55d0b6b0a93"
server_key_iv: "2714013aad5752ffa389983b0de2caf1"
server_key_version: "encryptionkey@0"
I20260619 14:57:50.875494 22514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:50.906396 22514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:50.910857 22514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:50.911160 22514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:50.912204 22514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:50.913619 22514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:50.913702 22514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:50.913866 22514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:50.913925 22514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:50.947610 22514 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:33021
I20260619 14:57:50.947650 22641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:33021 every 8 connection(s)
I20260619 14:57:50.948947 22514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:57:50.953239 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 22514
I20260619 14:57:50.953446 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:57:50.953866 11967 external_mini_cluster.cc:1428] Setting key 813482d9470a734ae2e6df77214120b9
Jun 19 14:57:50 dist-test-slave-76rk krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881070, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:50.969398 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:50.951514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:52385 (local address 127.11.175.254:36759)
0619 14:57:50.951965 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:50.951969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:50.952933 (+ 964us) server_negotiation.cc:407] Connection header received
0619 14:57:50.955056 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:50.955062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:50.955152 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:50.955271 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:50.956895 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:50.957629 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:50.959257 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:50.959677 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:50.959820 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:50.963349 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:50.963354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:50.963357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:50.963388 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:50.965936 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:50.966936 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:50.966940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:50.966942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:50.967071 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:50.967557 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:50.967560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:50.967562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:50.967816 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:50.967926 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:50.968823 (+ 897us) server_negotiation.cc:299] Negotiation successful
0619 14:57:50.968993 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":221,"threads_started":1}
I20260619 14:57:50.970894 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:50.952094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:36759 (local address 127.11.175.194:52385)
0619 14:57:50.952775 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:50.952812 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:50.954797 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:50.955548 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:50.955562 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:50.956161 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:50.956729 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:50.956746 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:50.957799 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:50.957803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:50.959104 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:50.959117 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:50.959904 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:50.959911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:50.960153 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:50.960835 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:50.960858 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:50.963129 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:50.966089 (+ 2960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:50.966100 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:50.966121 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:50.966817 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:50.967181 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:50.967186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:50.967192 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:50.967445 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:50.967950 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:50.967960 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:50.968429 (+ 469us) client_negotiation.cc:769] Sending connection context
0619 14:57:50.968729 (+ 300us) client_negotiation.cc:240] Negotiation successful
0619 14:57:50.969340 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":274,"threads_started":1}
I20260619 14:57:50.972540 22642 heartbeater.cc:347] Connected to a master server at 127.11.175.254:36759
I20260619 14:57:50.972862 22642 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:50.973590 22642 heartbeater.cc:511] Master 127.11.175.254:36759 requested a full tablet report, sending...
I20260619 14:57:50.975104 22319 ts_manager.cc:194] Registered new tserver with Master: d8937c88921142f89bcf7c6f3862217f (127.11.175.194:33021)
I20260619 14:57:50.975935 22319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:52385
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:57:51.007476 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36759
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:57:51.334205 22648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:57:51.334681 22648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:57:51.334856 22648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:57:51.342341 22648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:57:51.342454 22648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:57:51.342679 22648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:57:51.353268 22648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:57:51.354617 22648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:57:51.356019 22648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:57:51.369969 22653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:51.369947 22654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:57:51.370046 22656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:57:51.371542 22648 server_base.cc:1034] running on GCE node
I20260619 14:57:51.372303 22648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:57:51.373387 22648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:57:51.374594 22648 hybrid_clock.cc:648] HybridClock initialized: now 1781881071374546 us; error 43 us; skew 500 ppm
I20260619 14:57:51.375000 22648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:57:51 dist-test-slave-76rk krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:57:51.378307 22648 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:57:51.379988 22648 webserver.cc:460] Webserver started at http://127.11.175.195:46761/ using document root <none> and password file <none>
I20260619 14:57:51.380719 22648 fs_manager.cc:359] Metadata directory not provided
I20260619 14:57:51.380853 22648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:57:51.381168 22648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:57:51.384308 22648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "7e5db63661dc4b53ba32ec6d7b48b259"
format_stamp: "Formatted at 2026-06-19 14:57:51 on dist-test-slave-76rk"
server_key: "9a6ae4f7afd27de9caec5129f67408ff"
server_key_iv: "7880953a81678baea157353f7b8bb49c"
server_key_version: "encryptionkey@0"
I20260619 14:57:51.385185 22648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "7e5db63661dc4b53ba32ec6d7b48b259"
format_stamp: "Formatted at 2026-06-19 14:57:51 on dist-test-slave-76rk"
server_key: "9a6ae4f7afd27de9caec5129f67408ff"
server_key_iv: "7880953a81678baea157353f7b8bb49c"
server_key_version: "encryptionkey@0"
I20260619 14:57:51.390863 22648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 14:57:51.395032 22663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:51.396636 22648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:57:51.396844 22648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "7e5db63661dc4b53ba32ec6d7b48b259"
format_stamp: "Formatted at 2026-06-19 14:57:51 on dist-test-slave-76rk"
server_key: "9a6ae4f7afd27de9caec5129f67408ff"
server_key_iv: "7880953a81678baea157353f7b8bb49c"
server_key_version: "encryptionkey@0"
I20260619 14:57:51.397081 22648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:57:51.413156 22648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:57:51.418032 22648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:57:51.418350 22648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:57:51.419446 22648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:57:51.421010 22648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:57:51.421116 22648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:51.421268 22648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:57:51.421343 22648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:57:51.447189 22508 heartbeater.cc:503] Master 127.11.175.254:36759 was elected leader, sending a full tablet report...
I20260619 14:57:51.455018 22648 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:40749
I20260619 14:57:51.455060 22775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:40749 every 8 connection(s)
I20260619 14:57:51.456326 22648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:57:51.461916 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 22648
I20260619 14:57:51.462117 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:57:51.462574 11967 external_mini_cluster.cc:1428] Setting key b040cedd85f857c3e0c67b03dc5e22d5
Jun 19 14:57:51 dist-test-slave-76rk krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:57:51.476588 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.458837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:55513 (local address 127.11.175.254:36759)
0619 14:57:51.459049 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:51.459056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:51.460201 (+ 1145us) server_negotiation.cc:407] Connection header received
0619 14:57:51.461965 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:51.461972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:51.462067 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:51.462186 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:51.464054 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.465046 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.466947 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.467339 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.467436 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.470836 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:51.470841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:51.470844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:51.470878 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:51.473613 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:51.474332 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:51.474336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:51.474338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:51.474453 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:51.474928 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:51.474932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:51.474933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:51.475206 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:51.475324 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:51.476026 (+ 702us) server_negotiation.cc:299] Negotiation successful
0619 14:57:51.476286 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 14:57:51.477613 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.459351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:36759 (local address 127.11.175.195:55513)
0619 14:57:51.460031 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:51.460076 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:51.461690 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:51.462449 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:51.462458 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:51.463135 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:57:51.463833 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.463860 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.465224 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.465230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.466813 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.466834 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.467522 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.467527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.467739 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.468373 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:57:51.468395 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:57:51.470562 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:57:51.473771 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:51.473779 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:51.473793 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:51.474218 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:51.474545 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:57:51.474548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:57:51.474550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:57:51.474824 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:57:51.475312 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:57:51.475318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:57:51.475647 (+ 329us) client_negotiation.cc:769] Sending connection context
0619 14:57:51.475921 (+ 274us) client_negotiation.cc:240] Negotiation successful
0619 14:57:51.476311 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":266,"threads_started":1}
I20260619 14:57:51.479063 22776 heartbeater.cc:347] Connected to a master server at 127.11.175.254:36759
I20260619 14:57:51.479399 22776 heartbeater.cc:464] Registering TS with master...
I20260619 14:57:51.480192 22776 heartbeater.cc:511] Master 127.11.175.254:36759 requested a full tablet report, sending...
I20260619 14:57:51.481657 22319 ts_manager.cc:194] Registered new tserver with Master: 7e5db63661dc4b53ba32ec6d7b48b259 (127.11.175.195:40749)
I20260619 14:57:51.482415 22319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:55513
I20260619 14:57:51.490792 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 14:57:51.507823 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.495032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.11.175.254:36759)
0619 14:57:51.495253 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:51.495262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:51.495693 (+ 431us) server_negotiation.cc:407] Connection header received
0619 14:57:51.495824 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:51.495827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:51.495900 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:51.495980 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:51.497333 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.498054 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.499333 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.499741 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.499838 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.502462 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:51.502467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:51.502470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:51.502505 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:51.505215 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:51.505890 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:51.505894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:51.505896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:51.506021 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:51.506376 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:51.506378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:51.506380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:51.506753 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:51.506948 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:51.507281 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 14:57:51.507483 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 14:57:51.512476 22319 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:57:51.519760 22319 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:57:51.553583 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.531904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39184 (local address 127.11.175.193:33211)
0619 14:57:51.543517 (+ 11613us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:51.543531 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:51.544511 (+ 980us) server_negotiation.cc:407] Connection header received
0619 14:57:51.544977 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:51.544986 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:51.545294 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:51.545465 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:51.546737 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.548513 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.550997 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.552419 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.552636 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.552834 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:51.553066 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 14:57:51.553225 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11355,"thread_start_us":11623,"threads_started":1}
I20260619 14:57:51.553654 22788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.531608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:33211 (local address 127.0.0.1:39184)
0619 14:57:51.544347 (+ 12739us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:51.544386 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:51.544673 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:51.545800 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:51.545807 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:51.545894 (+ 87us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:51.546446 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.546462 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.548669 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.548674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.550845 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.550857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.552632 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.552638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.552821 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.552877 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 14:57:51.553011 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 14:57:51.553220 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12491,"mutex_wait_us":47,"thread_start_us":12340,"threads_started":1}
I20260619 14:57:51.554688 22792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.531902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:33021 (local address 127.0.0.1:36196)
0619 14:57:51.545052 (+ 13150us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:51.545068 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:51.545164 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:51.547017 (+ 1853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:51.547025 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:51.547054 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:51.547456 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.547470 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.549980 (+ 2510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.549985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.552192 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.552205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.554010 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.554017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.554184 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.554222 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 14:57:51.554281 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 14:57:51.554363 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13033,"thread_start_us":14635,"threads_started":1}
I20260619 14:57:51.555135 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.533520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:40749 (local address 127.0.0.1:57172)
0619 14:57:51.546829 (+ 13309us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:51.546843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:51.546986 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:51.547599 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:51.547604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:51.547629 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:51.548055 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.548066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.549599 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.549604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.551913 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.551927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.554296 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.554304 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.554474 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.554506 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 14:57:51.554603 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 14:57:51.554753 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13206,"mutex_wait_us":25,"thread_start_us":11292,"threads_started":1}
I20260619 14:57:51.555210 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.533823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57172 (local address 127.11.175.195:40749)
0619 14:57:51.545965 (+ 12142us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:51.545973 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:51.546945 (+ 972us) server_negotiation.cc:407] Connection header received
0619 14:57:51.547151 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:51.547156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:51.547417 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:51.547617 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:51.548250 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.549453 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.552093 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.553515 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.553697 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.554123 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:51.554713 (+ 590us) server_negotiation.cc:299] Negotiation successful
0619 14:57:51.554886 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11962,"thread_start_us":11725,"threads_started":1}
I20260619 14:57:51.555871 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.532413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36196 (local address 127.11.175.194:33021)
0619 14:57:51.546384 (+ 13971us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:51.546395 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:51.546424 (+ 29us) server_negotiation.cc:407] Connection header received
0619 14:57:51.546524 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:51.546533 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:51.546795 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:51.548009 (+ 1214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:51.548225 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.549840 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.552360 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.553779 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.555114 (+ 1335us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.555305 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:51.555384 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 14:57:51.555551 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13810,"thread_start_us":13686,"threads_started":1}
I20260619 14:57:51.558053 22443 tablet_service.cc:1467] Processing CreateTablet for tablet 1a6ade93ed834bdc93d1b39569f3903e (DEFAULT_TABLE table=test-table [id=f53c51e19ea148a39a139990c47f53c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:51.559401 22443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a6ade93ed834bdc93d1b39569f3903e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:51.558859 22711 tablet_service.cc:1467] Processing CreateTablet for tablet 1a6ade93ed834bdc93d1b39569f3903e (DEFAULT_TABLE table=test-table [id=f53c51e19ea148a39a139990c47f53c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:51.559255 22577 tablet_service.cc:1467] Processing CreateTablet for tablet 1a6ade93ed834bdc93d1b39569f3903e (DEFAULT_TABLE table=test-table [id=f53c51e19ea148a39a139990c47f53c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:57:51.560277 22711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a6ade93ed834bdc93d1b39569f3903e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:51.560426 22577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a6ade93ed834bdc93d1b39569f3903e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:57:51.571399 22794 tablet_bootstrap.cc:492] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: Bootstrap starting.
I20260619 14:57:51.574774 22794 tablet_bootstrap.cc:654] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:51.576102 22796 tablet_bootstrap.cc:492] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: Bootstrap starting.
I20260619 14:57:51.576359 22794 log.cc:824] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:51.579389 22795 tablet_bootstrap.cc:492] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: Bootstrap starting.
I20260619 14:57:51.580651 22796 tablet_bootstrap.cc:654] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:51.582209 22796 log.cc:824] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:51.582289 22795 tablet_bootstrap.cc:654] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: Neither blocks nor log segments found. Creating new log.
I20260619 14:57:51.583755 22795 log.cc:824] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: Log is configured to *not* fsync() on all Append() calls
I20260619 14:57:51.585117 22794 tablet_bootstrap.cc:492] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: No bootstrap required, opened a new log
I20260619 14:57:51.585413 22794 ts_tablet_manager.cc:1397] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260619 14:57:51.586225 22795 tablet_bootstrap.cc:492] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: No bootstrap required, opened a new log
I20260619 14:57:51.586462 22795 ts_tablet_manager.cc:1397] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 14:57:51.587213 22796 tablet_bootstrap.cc:492] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: No bootstrap required, opened a new log
I20260619 14:57:51.587456 22796 ts_tablet_manager.cc:1397] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260619 14:57:51.593693 22795 raft_consensus.cc:348] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 } } peers { permanent_uuid: "d8937c88921142f89bcf7c6f3862217f" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33021 } } peers { permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 } }
I20260619 14:57:51.594166 22795 raft_consensus.cc:374] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:51.594287 22795 raft_consensus.cc:729] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8937c88921142f89bcf7c6f3862217f, State: Initialized, Role: FOLLOWER
I20260619 14:57:51.595090 22795 consensus_queue.cc:260] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 } } peers { permanent_uuid: "d8937c88921142f89bcf7c6f3862217f" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33021 } } peers { permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 } }
I20260619 14:57:51.596253 22642 heartbeater.cc:503] Master 127.11.175.254:36759 was elected leader, sending a full tablet report...
I20260619 14:57:51.596318 22794 raft_consensus.cc:348] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 } } peers { permanent_uuid: "d8937c88921142f89bcf7c6f3862217f" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33021 } } peers { permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 } }
I20260619 14:57:51.596772 22794 raft_consensus.cc:374] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:51.596858 22794 raft_consensus.cc:729] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e5035beba3342eb956aeea91a613760, State: Initialized, Role: FOLLOWER
I20260619 14:57:51.596997 22795 ts_tablet_manager.cc:1428] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260619 14:57:51.597538 22794 consensus_queue.cc:260] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 } } peers { permanent_uuid: "d8937c88921142f89bcf7c6f3862217f" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33021 } } peers { permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 } }
I20260619 14:57:51.598098 22796 raft_consensus.cc:348] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 } } peers { permanent_uuid: "d8937c88921142f89bcf7c6f3862217f" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33021 } } peers { permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 } }
I20260619 14:57:51.598548 22796 raft_consensus.cc:374] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:57:51.598666 22796 raft_consensus.cc:729] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e5db63661dc4b53ba32ec6d7b48b259, State: Initialized, Role: FOLLOWER
I20260619 14:57:51.599257 22796 consensus_queue.cc:260] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 } } peers { permanent_uuid: "d8937c88921142f89bcf7c6f3862217f" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33021 } } peers { permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 } }
I20260619 14:57:51.599511 22794 ts_tablet_manager.cc:1428] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260619 14:57:51.600943 22796 ts_tablet_manager.cc:1428] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260619 14:57:51.601496 22776 heartbeater.cc:503] Master 127.11.175.254:36759 was elected leader, sending a full tablet report...
I20260619 14:57:51.604185 22800 raft_consensus.cc:482] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:57:51.604451 22800 raft_consensus.cc:504] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 } } peers { permanent_uuid: "d8937c88921142f89bcf7c6f3862217f" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33021 } } peers { permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 } }
I20260619 14:57:51.606047 22800 leader_election.cc:288] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e5035beba3342eb956aeea91a613760 (127.11.175.193:33211), 7e5db63661dc4b53ba32ec6d7b48b259 (127.11.175.195:40749)
I20260619 14:57:51.614127 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.606755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:53689 (local address 127.11.175.193:33211)
0619 14:57:51.607005 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:51.607012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:51.607360 (+ 348us) server_negotiation.cc:407] Connection header received
0619 14:57:51.607623 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:51.607629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:51.607707 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:51.607801 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:51.608412 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.609397 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.611991 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.613208 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.613377 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.613455 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:51.613774 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 14:57:51.613870 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 14:57:51.614370 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.607137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:58123 (local address 127.11.175.195:40749)
0619 14:57:51.607334 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:51.607341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:51.607902 (+ 561us) server_negotiation.cc:407] Connection header received
0619 14:57:51.608077 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:51.608081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:51.608132 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:51.608206 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:57:51.608749 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.609745 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.612148 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.613405 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.613549 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.613596 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:51.613965 (+ 369us) server_negotiation.cc:299] Negotiation successful
0619 14:57:51.614087 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 14:57:51.614473 22803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.606459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:40749 (local address 127.11.175.194:58123)
0619 14:57:51.607815 (+ 1356us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:51.607831 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:51.607947 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:51.608253 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:51.608259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:51.608281 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:51.608613 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.608621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.609868 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.609873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.612008 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.612019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.613598 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.613603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.613816 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.613844 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 14:57:51.613923 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 14:57:51.614114 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1217,"mutex_wait_us":75,"thread_start_us":725,"threads_started":1}
I20260619 14:57:51.614526 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.606344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:33211 (local address 127.11.175.194:53689)
0619 14:57:51.607254 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0619 14:57:51.607278 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 14:57:51.607466 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:57:51.607842 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:57:51.607850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:57:51.607882 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:57:51.608272 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.608282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.609518 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.609522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.611841 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:57:51.611856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.613394 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.613398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:57:51.613582 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.613639 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 14:57:51.613740 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 14:57:51.613878 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":25,"thread_start_us":318,"threads_started":1}
I20260619 14:57:51.615422 22731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a6ade93ed834bdc93d1b39569f3903e" candidate_uuid: "d8937c88921142f89bcf7c6f3862217f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" is_pre_election: true
I20260619 14:57:51.615423 22463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a6ade93ed834bdc93d1b39569f3903e" candidate_uuid: "d8937c88921142f89bcf7c6f3862217f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e5035beba3342eb956aeea91a613760" is_pre_election: true
I20260619 14:57:51.615947 22731 raft_consensus.cc:2454] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8937c88921142f89bcf7c6f3862217f in term 0.
I20260619 14:57:51.615952 22463 raft_consensus.cc:2454] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8937c88921142f89bcf7c6f3862217f in term 0.
I20260619 14:57:51.616575 22531 leader_election.cc:302] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1e5035beba3342eb956aeea91a613760, 7e5db63661dc4b53ba32ec6d7b48b259, d8937c88921142f89bcf7c6f3862217f; no voters:
I20260619 14:57:51.616974 22800 raft_consensus.cc:2781] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:57:51.617117 22800 raft_consensus.cc:482] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:57:51.617242 22800 raft_consensus.cc:3037] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:51.619567 22800 raft_consensus.cc:504] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 } } peers { permanent_uuid: "d8937c88921142f89bcf7c6f3862217f" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33021 } } peers { permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 } }
I20260619 14:57:51.620307 22800 leader_election.cc:288] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [CANDIDATE]: Term 1 election: Requested vote from peers 1e5035beba3342eb956aeea91a613760 (127.11.175.193:33211), 7e5db63661dc4b53ba32ec6d7b48b259 (127.11.175.195:40749)
I20260619 14:57:51.620741 22463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a6ade93ed834bdc93d1b39569f3903e" candidate_uuid: "d8937c88921142f89bcf7c6f3862217f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e5035beba3342eb956aeea91a613760"
I20260619 14:57:51.620959 22463 raft_consensus.cc:3037] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:51.621163 22731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a6ade93ed834bdc93d1b39569f3903e" candidate_uuid: "d8937c88921142f89bcf7c6f3862217f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e5db63661dc4b53ba32ec6d7b48b259"
I20260619 14:57:51.621407 22731 raft_consensus.cc:3037] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:57:51.623540 22463 raft_consensus.cc:2454] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8937c88921142f89bcf7c6f3862217f in term 1.
I20260619 14:57:51.623986 22532 leader_election.cc:302] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e5035beba3342eb956aeea91a613760, d8937c88921142f89bcf7c6f3862217f; no voters:
I20260619 14:57:51.624101 22731 raft_consensus.cc:2454] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8937c88921142f89bcf7c6f3862217f in term 1.
I20260619 14:57:51.624351 22800 raft_consensus.cc:2781] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:57:51.624838 22800 raft_consensus.cc:686] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 1 LEADER]: Becoming Leader. State: Replica: d8937c88921142f89bcf7c6f3862217f, State: Running, Role: LEADER
I20260619 14:57:51.625301 22800 consensus_queue.cc:237] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 } } peers { permanent_uuid: "d8937c88921142f89bcf7c6f3862217f" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33021 } } peers { permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 } }
I20260619 14:57:51.630323 22317 catalog_manager.cc:5495] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f reported cstate change: term changed from 0 to 1, leader changed from <none> to d8937c88921142f89bcf7c6f3862217f (127.11.175.194). New cstate: current_term: 1 leader_uuid: "d8937c88921142f89bcf7c6f3862217f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8937c88921142f89bcf7c6f3862217f" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260619 14:57:51.656306 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.651352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.11.175.194:33021)
0619 14:57:51.651542 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:51.651546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:51.651625 (+ 79us) server_negotiation.cc:407] Connection header received
0619 14:57:51.651797 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:51.651801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:51.651877 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:51.652005 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:57:51.652335 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.653051 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.654317 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.654785 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.654883 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.655302 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:57:51.655812 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:57:51.655922 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:51.655999 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 14:57:51.656097 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 14:57:51.664352 22731 raft_consensus.cc:1261] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259 [term 1 FOLLOWER]: Refusing update from remote peer d8937c88921142f89bcf7c6f3862217f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:57:51.664352 22463 raft_consensus.cc:1261] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760 [term 1 FOLLOWER]: Refusing update from remote peer d8937c88921142f89bcf7c6f3862217f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260619 14:57:51.665498 22509 tablet.cc:2369] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:57:51.665395 22805 consensus_queue.cc:1035] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e5035beba3342eb956aeea91a613760" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 33211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:51.665783 22800 consensus_queue.cc:1035] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e5db63661dc4b53ba32ec6d7b48b259" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:57:51.677129 22808 mvcc.cc:205] Tried to move back new op lower bound from 7298584869526753280 to 7298584869381283840. Current Snapshot: MvccSnapshot[applied={T|T < 7298584869526753280}]
I20260619 14:57:51.693195 22319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:51.693482 22319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:57:51.697013 22319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42327c647a7c45e0a608cd2ee5c1c343: Sending DeleteTablet for 3 replicas of tablet 1a6ade93ed834bdc93d1b39569f3903e
I20260619 14:57:51.698156 22443 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a6ade93ed834bdc93d1b39569f3903e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:51 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:39184
I20260619 14:57:51.698163 22577 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a6ade93ed834bdc93d1b39569f3903e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:51 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:36196
I20260619 14:57:51.698441 22711 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a6ade93ed834bdc93d1b39569f3903e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:57:51 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:57172
I20260619 14:57:51.699865 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 22380
I20260619 14:57:51.707211 22818 tablet_replica.cc:330] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: stopping tablet replica
I20260619 14:57:51.707770 22818 raft_consensus.cc:2229] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:51.708453 22818 raft_consensus.cc:2258] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:51.714826 22819 tablet_replica.cc:330] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: stopping tablet replica
I20260619 14:57:51.715076 22818 ts_tablet_manager.cc:1905] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:51.715312 22819 raft_consensus.cc:2229] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:57:51.716053 22819 raft_consensus.cc:2258] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:51.721987 22819 ts_tablet_manager.cc:1905] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 14:57:51 dist-test-slave-76rk krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:57:51.724682 22820 tablet_replica.cc:330] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: stopping tablet replica
I20260619 14:57:51.725126 22818 ts_tablet_manager.cc:1918] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:51.725147 22820 raft_consensus.cc:2229] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:57:51.725304 22818 log.cc:1196] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/1a6ade93ed834bdc93d1b39569f3903e
I20260619 14:57:51.725886 22818 ts_tablet_manager.cc:1939] T 1a6ade93ed834bdc93d1b39569f3903e P 1e5035beba3342eb956aeea91a613760: Deleting consensus metadata
I20260619 14:57:51.727852 22305 catalog_manager.cc:4841] TS 1e5035beba3342eb956aeea91a613760 (127.11.175.193:33211): tablet 1a6ade93ed834bdc93d1b39569f3903e (table test-table [id=f53c51e19ea148a39a139990c47f53c1]) successfully deleted
I20260619 14:57:51.730211 22820 raft_consensus.cc:2258] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:57:51.731655 22819 ts_tablet_manager.cc:1918] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:51.731856 22819 log.cc:1196] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/1a6ade93ed834bdc93d1b39569f3903e
I20260619 14:57:51.732419 22819 ts_tablet_manager.cc:1939] T 1a6ade93ed834bdc93d1b39569f3903e P d8937c88921142f89bcf7c6f3862217f: Deleting consensus metadata
I20260619 14:57:51.732564 22820 ts_tablet_manager.cc:1905] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:57:51.733373 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.702506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.11.175.193:33211)
0619 14:57:51.702757 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:51.702767 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:51.707459 (+ 4692us) server_negotiation.cc:407] Connection header received
0619 14:57:51.709005 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:51.709012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:51.709102 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:51.709181 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:51.711241 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.712419 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.716441 (+ 4022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.717182 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.717301 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.725599 (+ 8298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:51.725615 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:51.725633 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:51.725678 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:51.729771 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:51.730730 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:51.730739 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:51.730743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:51.730899 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:51.731380 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:51.731388 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:51.731394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:51.731864 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:51.732189 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:51.732782 (+ 593us) server_negotiation.cc:299] Negotiation successful
0619 14:57:51.733008 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 14:57:51.734236 22303 catalog_manager.cc:4841] TS d8937c88921142f89bcf7c6f3862217f (127.11.175.194:33021): tablet 1a6ade93ed834bdc93d1b39569f3903e (table test-table [id=f53c51e19ea148a39a139990c47f53c1]) successfully deleted
I20260619 14:57:51.735471 22503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:51.739545 22820 ts_tablet_manager.cc:1918] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:57:51.739739 22820 log.cc:1196] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/1a6ade93ed834bdc93d1b39569f3903e
I20260619 14:57:51.740520 22820 ts_tablet_manager.cc:1939] T 1a6ade93ed834bdc93d1b39569f3903e P 7e5db63661dc4b53ba32ec6d7b48b259: Deleting consensus metadata
I20260619 14:57:51.743304 22304 catalog_manager.cc:4841] TS 7e5db63661dc4b53ba32ec6d7b48b259 (127.11.175.195:40749): tablet 1a6ade93ed834bdc93d1b39569f3903e (table test-table [id=f53c51e19ea148a39a139990c47f53c1]) successfully deleted
I20260619 14:57:51.859762 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 22380
I20260619 14:57:51.877632 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 22514
Jun 19 14:57:51 dist-test-slave-76rk krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:57:51.895300 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:51.878599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36210 (local address 127.11.175.194:33021)
0619 14:57:51.878832 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:51.878838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:51.878860 (+ 22us) server_negotiation.cc:407] Connection header received
0619 14:57:51.878926 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:51.878931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:51.879012 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:51.879138 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:51.880468 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.881633 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.883109 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:51.883808 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:51.883947 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:51.888138 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:51.888159 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:51.888187 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:51.888243 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:51.892261 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:51.892992 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:51.893002 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:51.893007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:51.893183 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:51.893545 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:51.893550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:51.893553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:51.894035 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:51.894379 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:51.894663 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 14:57:51.894906 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 14:57:51.896091 22637 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:52.010953 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 22514
I20260619 14:57:52.030578 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 22648
Jun 19 14:57:52 dist-test-slave-76rk krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:57:52.045559 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:57:52.031398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.11.175.195:40749)
0619 14:57:52.031560 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 14:57:52.031565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:57:52.031582 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:57:52.031728 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:57:52.031732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:57:52.031794 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:57:52.031882 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:57:52.033134 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:52.033835 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:52.035203 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:57:52.035594 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:57:52.035686 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:57:52.040016 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:57:52.040032 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:57:52.040047 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:57:52.040087 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:57:52.042825 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:52.043486 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:52.043492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:52.043495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:52.043631 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:57:52.043937 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:57:52.043940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:57:52.043942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:57:52.044294 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:57:52.044826 (+ 532us) server_negotiation.cc:1035] Waiting for connection context
0619 14:57:52.045130 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 14:57:52.045317 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 14:57:52.046376 22771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:52.137045 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 22648
I20260619 14:57:52.153815 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 22287
I20260619 14:57:52.154994 22349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:57:52.264704 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 22287
2026-06-19T14:57:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4293 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:57:52 dist-test-slave-76rk krb5kdc[22828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:57:52 dist-test-slave-76rk krb5kdc[22828](info): set up 2 sockets
Jun 19 14:57:52 dist-test-slave-76rk krb5kdc[22828](info): commencing operation
krb5kdc: starting...
W20260619 14:57:53.375756 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:57:53 dist-test-slave-76rk krb5kdc[22828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:57:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:57:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.175.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.175.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/rangeradmin_127.11.175.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/rangeradmin_127.11.175.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.175.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.175.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/rangerlookup_127.11.175.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/rangerlookup_127.11.175.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.175.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.175.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/HTTP_127.11.175.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/HTTP_127.11.175.212@KRBTEST.COM.keytab.
I20260619 14:57:53.713467 11967 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDoOfXv/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/postgres -l logfile start
2026-06-19 14:57:56.250 UTC [22858] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 14:57:56.250 UTC [22858] LOG: listening on IPv4 address "127.11.175.212", port 42163
2026-06-19 14:57:56.257 UTC [22858] LOG: listening on Unix socket "/tmp/.s.PGSQL.42163"
2026-06-19 14:57:56.280 UTC [22861] LOG: database system was shut down at 2026-06-19 14:57:54 UTC
2026-06-19 14:57:56.284 UTC [22858] LOG: database system is ready to accept connections
I20260619 14:57:57.267300 11967 mini_postgres.cc:96] Postgres bound to 42163
2026-06-19 14:57:57.287 UTC [22869] FATAL: database "slave" does not exist
127.11.175.212:42163 - accepting connections
I20260619 14:57:57.288692 11967 mini_ranger.cc:161] Starting Ranger...
I20260619 14:57:57.317355 11967 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 14:57:57.815415 11967 mini_ranger.cc:87] Created ranger Postgres database
I20260619 14:57:57.815524 11967 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-admin
2026-06-19 14:57:58,183 [I] DB FLAVOR :POSTGRES
2026-06-19 14:57:58,183 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 14:57:58,184 [I] Checking connection..
2026-06-19 14:57:58,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 14:57:58,481 [I] Checking connection passed.
2026-06-19 14:57:58,481 [I] --------- Verifying version history table ---------
2026-06-19 14:57:58,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 14:57:58,735 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 14:57:58,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 14:57:58,990 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 14:57:58,991 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 14:57:58,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 14:57:59.246 UTC [22933] WARNING: there is no transaction in progress
2026-06-19 14:57:59,253 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 14:57:59,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 14:57:59,514 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 14:57:59,514 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 14:57:59,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 14:57:59,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 14:58:00,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-76rk.c.gcp-upstream.internal','N') ;"
2026-06-19 14:58:00,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 14:58:00,486 [I] Table x_portal_user does not exist in database ranger
2026-06-19 14:58:00,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 14:58:00,733 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 14:58:00,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 14:58:00,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 14:58:01,259 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 14:58:01,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 14:58:02.236 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.252 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.268 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.283 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.300 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.319 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.376 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.383 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.397 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.408 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.418 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.431 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.441 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.447 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.453 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:02.459 UTC [23041] WARNING: there is no transaction in progress
2026-06-19 14:58:03,108 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 14:58:03,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 14:58:03,336 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 14:58:03,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 14:58:03,599 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 14:58:03,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 14:58:03,852 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 14:58:03,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 14:58:04,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 14:58:04,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-76rk.c.gcp-upstream.internal';"
2026-06-19 14:58:04,645 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 14:58:04,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 14:58:04,902 [I] DB_PATCHES have already been applied
I20260619 14:58:04.908968 11967 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDoOfXv/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 14:58:04.909062 11967 mini_ranger.cc:193] Using host: 127.11.175.212
I20260619 14:58:04.925937 11967 mini_ranger.cc:227] Ranger admin URL: http://127.11.175.212:37269
Jun 19, 2026 2:58:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:58:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:58:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:58:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:58:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:58:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:58:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 14:58:06.077968 11967 mini_ranger.cc:160] Time spent starting Ranger: real 8.789s user 0.000s sys 0.046s
Jun 19, 2026 2:58:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:58:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 2:58:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:58:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:58:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 14:58:29.976388 11967 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.175.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.175.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/rangerkms_127.11.175.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/rangerkms_127.11.175.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.175.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.175.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.175.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/HTTP_127.11.175.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.175.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/HTTP_127.11.175.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 14:58:30 dist-test-slave-76rk krb5kdc[22828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881110, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 14:58:30.185457 11967 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 14:58:30.209010 11967 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 14:58:31.003748 11967 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 14:58:31.003961 11967 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-kms
I20260619 14:58:31.004108 11967 mini_ranger_kms.cc:227] Using postgres at 127.11.175.212:42163
2026-06-19 14:58:31,765 [I] DB FLAVOR :POSTGRES
2026-06-19 14:58:31,765 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 14:58:31,765 [I] Checking connection
2026-06-19 14:58:31,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 14:58:32,042 [I] connection success
2026-06-19 14:58:32,042 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 14:58:32,042 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 14:58:32,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 14:58:32,309 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 14:58:32,310 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 14:58:32,310 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 14:58:32,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:42163/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 14:58:32,596 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 14:58:33.136847 11967 mini_ranger_kms.cc:310] Created kms service
I20260619 14:58:33.324532 11967 mini_ranger_kms.cc:326] Created kudu user
I20260619 14:58:33.368479 11967 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 14:58:33 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881110, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:58:33.787035 11967 mini_ranger_kms.cc:384] Added ranger policy
I20260619 14:58:33.787390 11967 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 14:58:33.787470 11967 mini_ranger_kms.cc:240] Using host: 127.11.175.212
I20260619 14:58:33.811820 11967 mini_ranger_kms.cc:276] Ranger KMS PID: 23222
I20260619 14:58:33.812027 11967 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.175.212:42899
Jun 19, 2026 2:58:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:58:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:58:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:58:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:58:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:58:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 19 14:58:35 dist-test-slave-76rk krb5kdc[22828](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781881115, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.175.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:58:36.211357 11967 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.026s user 0.149s sys 0.365s
I20260619 14:58:36.211637 11967 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 14:58:36.211876 11967 mini_ranger_kms.cc:401] 127.11.175.212:42899/kms/v1/keys
Jun 19 14:58:36 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881115, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.175.212@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
Jun 19 14:58:36 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881115, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.175.212@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
Jun 19 14:58:39 dist-test-slave-76rk krb5kdc[22828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:58:39.086824 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38677
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38677
--ranger_config_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:42899/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:58:39.437264 23272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:58:39.437570 23272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:58:39.437731 23272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:58:39.445273 23272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:58:39.445396 23272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:58:39.445501 23272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:58:39.445595 23272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:58:39.445676 23272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:58:39.455992 23272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.175.212:42899/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38677
--ranger_config_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:58:39.457360 23272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:58:39.458943 23272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:58:39.472182 23277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:58:39.472113 23278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:58:39.472152 23280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:58:39.472544 23272 server_base.cc:1034] running on GCE node
I20260619 14:58:39.473767 23272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:58:39.475334 23272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:58:39.476578 23272 hybrid_clock.cc:648] HybridClock initialized: now 1781881119476544 us; error 31 us; skew 500 ppm
I20260619 14:58:39.476908 23272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:58:39 dist-test-slave-76rk krb5kdc[22828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881119, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:58:39.483031 23272 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:58:39.484781 23272 webserver.cc:460] Webserver started at http://127.11.175.254:43707/ using document root <none> and password file <none>
I20260619 14:58:39.485513 23272 fs_manager.cc:359] Metadata directory not provided
I20260619 14:58:39.485625 23272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:58:39.486006 23272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 14:58:39 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881119, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:58:39.539726 23272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "374a619379d64f92991cf301716902ad"
format_stamp: "Formatted at 2026-06-19 14:58:39 on dist-test-slave-76rk"
server_key: "f88f005fd6f673c48228146eedac7028"
server_key_iv: "f802bb0e1c34ba18e956a29258524966"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:39.540889 23272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "374a619379d64f92991cf301716902ad"
format_stamp: "Formatted at 2026-06-19 14:58:39 on dist-test-slave-76rk"
server_key: "f88f005fd6f673c48228146eedac7028"
server_key_iv: "f802bb0e1c34ba18e956a29258524966"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:39.548456 23272 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260619 14:58:39.570117 23287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:58:39.572163 23272 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260619 14:58:39.572383 23272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "374a619379d64f92991cf301716902ad"
format_stamp: "Formatted at 2026-06-19 14:58:39 on dist-test-slave-76rk"
server_key: "f88f005fd6f673c48228146eedac7028"
server_key_iv: "f802bb0e1c34ba18e956a29258524966"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:39.572660 23272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:58:39.612565 23272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:58:39.618403 23272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:58:39.618832 23272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:58:39.640964 23272 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:38677
I20260619 14:58:39.640966 23338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:38677 every 8 connection(s)
I20260619 14:58:39.642602 23272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:58:39.646845 23339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:58:39.647320 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 23272
I20260619 14:58:39.647590 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:58:39.648478 11967 external_mini_cluster.cc:1428] Setting key d2a52a75fcdc59eea8023e44c7865a02
I20260619 14:58:39.664956 23339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad: Bootstrap starting.
Jun 19 14:58:39 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:58:39.669416 23339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad: Neither blocks nor log segments found. Creating new log.
I20260619 14:58:39.670524 23339 log.cc:824] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad: Log is configured to *not* fsync() on all Append() calls
I20260619 14:58:39.674490 23339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad: No bootstrap required, opened a new log
I20260619 14:58:39.680030 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:39.651164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.11.175.254:38677)
0619 14:58:39.652397 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:39.652421 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:39.652492 (+ 71us) server_negotiation.cc:407] Connection header received
0619 14:58:39.655489 (+ 2997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:39.655530 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:39.656234 (+ 704us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:39.656823 (+ 589us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:58:39.658766 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:39.661166 (+ 2400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:39.663161 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:39.663934 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:39.664078 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:39.670011 (+ 5933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:58:39.670021 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:58:39.670038 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:58:39.670085 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:58:39.674159 (+ 4074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:39.675217 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:39.675241 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:39.675251 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:39.675504 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:39.676075 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:39.676080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:39.676083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:39.676819 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:58:39.677168 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:39.677685 (+ 517us) server_negotiation.cc:299] Negotiation successful
0619 14:58:39.678175 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":928,"thread_start_us":486,"threads_started":1}
I20260619 14:58:39.685942 23339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374a619379d64f92991cf301716902ad" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38677 } }
I20260619 14:58:39.686406 23339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:58:39.686520 23339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374a619379d64f92991cf301716902ad, State: Initialized, Role: FOLLOWER
I20260619 14:58:39.687175 23339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374a619379d64f92991cf301716902ad" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38677 } }
I20260619 14:58:39.687422 23339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:58:39.687557 23339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:58:39.687731 23339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:58:39.689937 23339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374a619379d64f92991cf301716902ad" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38677 } }
I20260619 14:58:39.690485 23339 leader_election.cc:302] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 374a619379d64f92991cf301716902ad; no voters:
I20260619 14:58:39.691143 23339 leader_election.cc:288] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:58:39.691460 23344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:58:39.692724 23344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [term 1 LEADER]: Becoming Leader. State: Replica: 374a619379d64f92991cf301716902ad, State: Running, Role: LEADER
I20260619 14:58:39.693323 23344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374a619379d64f92991cf301716902ad" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38677 } }
I20260619 14:58:39.693774 23339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:58:39.696547 23346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 374a619379d64f92991cf301716902ad. Latest consensus state: current_term: 1 leader_uuid: "374a619379d64f92991cf301716902ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374a619379d64f92991cf301716902ad" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38677 } } }
I20260619 14:58:39.697142 23345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "374a619379d64f92991cf301716902ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374a619379d64f92991cf301716902ad" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38677 } } }
I20260619 14:58:39.697609 23346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [sys.catalog]: This master's current role is: LEADER
I20260619 14:58:39.697605 23345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad [sys.catalog]: This master's current role is: LEADER
I20260619 14:58:39.707894 23353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:58:39.713183 23353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:58:39.714078 23339 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 14:58:39.734989 23353 catalog_manager.cc:1347] Generated new cluster ID: 7e092c7aa82b496b9b976cbbc9c5594b
I20260619 14:58:39.735162 23353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:58:39.750249 23353 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:58:39.752365 23353 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:58:39.764760 23353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad: Generated new TSK 0
I20260619 14:58:39.765955 23353 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:58:40.545838 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:42899/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38677
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 14:58:40 dist-test-slave-76rk krb5kdc[22828](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781881120, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 14:58:41.006480 23384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:58:41.007051 23384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:58:41.007227 23384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:58:41.019706 23384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:58:41.019881 23384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:58:41.020094 23384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:58:41.036973 23384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.175.212:42899/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:58:41.038825 23384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:58:41.040659 23384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:58:41.064644 23394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:58:41.072036 23395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:58:41.073110 23397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:58:41.073217 23384 server_base.cc:1034] running on GCE node
I20260619 14:58:41.074223 23384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:58:41.086963 23384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:58:41.088968 23384 hybrid_clock.cc:648] HybridClock initialized: now 1781881121088846 us; error 128 us; skew 500 ppm
I20260619 14:58:41.089497 23384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:58:41 dist-test-slave-76rk krb5kdc[22828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881121, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:58:41.093942 23384 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:58:41.100247 23384 webserver.cc:460] Webserver started at http://127.11.175.193:45775/ using document root <none> and password file <none>
I20260619 14:58:41.101207 23384 fs_manager.cc:359] Metadata directory not provided
I20260619 14:58:41.101423 23384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:58:41.101905 23384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 14:58:41 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881121, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:58:41.128903 23384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "9fef2aabb8454d3fa53f529e9aab9492"
format_stamp: "Formatted at 2026-06-19 14:58:41 on dist-test-slave-76rk"
server_key: "1527ddb801c3b1f2e6af269ec7845fce"
server_key_iv: "16bdd5b6d137a89fc53715a54b4eb651"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:41.131299 23384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "9fef2aabb8454d3fa53f529e9aab9492"
format_stamp: "Formatted at 2026-06-19 14:58:41 on dist-test-slave-76rk"
server_key: "1527ddb801c3b1f2e6af269ec7845fce"
server_key_iv: "16bdd5b6d137a89fc53715a54b4eb651"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:41.157402 23384 fs_manager.cc:691] Time spent creating directory manager: real 0.025s user 0.012s sys 0.008s
I20260619 14:58:41.188673 23404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:58:41.191190 23384 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.001s
I20260619 14:58:41.191552 23384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "9fef2aabb8454d3fa53f529e9aab9492"
format_stamp: "Formatted at 2026-06-19 14:58:41 on dist-test-slave-76rk"
server_key: "1527ddb801c3b1f2e6af269ec7845fce"
server_key_iv: "16bdd5b6d137a89fc53715a54b4eb651"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:41.192054 23384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:58:41.212412 23384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:58:41.217545 23384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:58:41.217934 23384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:58:41.219027 23384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:58:41.225443 23384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:58:41.225639 23384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:58:41.225871 23384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:58:41.225975 23384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:58:41.309509 23384 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:42137
I20260619 14:58:41.309592 23516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:42137 every 8 connection(s)
I20260619 14:58:41.311440 23384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:58:41.313926 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 23384
I20260619 14:58:41.314206 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:58:41.315311 11967 external_mini_cluster.cc:1428] Setting key 3f0df7922be99bd8cc850cb4edae75e4
Jun 19 14:58:41 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881121, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:58:41.347296 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:41.314775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:53201 (local address 127.11.175.254:38677)
0619 14:58:41.326784 (+ 12009us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:41.326790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:41.326810 (+ 20us) server_negotiation.cc:407] Connection header received
0619 14:58:41.326871 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:41.326876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:41.326974 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:41.327087 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:58:41.329501 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:41.330712 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:41.333381 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:41.334043 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:41.334150 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:41.339071 (+ 4921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:58:41.339080 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:58:41.339085 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:58:41.339132 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:58:41.343001 (+ 3869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:41.344133 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:41.344139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:41.344142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:41.344315 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:41.344956 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:41.344962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:41.344966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:41.345368 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:58:41.345514 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:41.346610 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0619 14:58:41.346904 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11876,"thread_start_us":11714,"threads_started":1}
I20260619 14:58:41.349867 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:41.315632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38677 (local address 127.11.175.193:53201)
0619 14:58:41.323434 (+ 7802us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:41.323490 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 14:58:41.326017 (+ 2527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:58:41.327575 (+ 1558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:58:41.327588 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:58:41.328297 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:58:41.329232 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:41.329271 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:41.330943 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:41.330948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:41.333144 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:41.333174 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:41.334610 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:41.334618 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:41.334969 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:41.335847 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:58:41.335880 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:58:41.338761 (+ 2881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:58:41.343216 (+ 4455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:58:41.343226 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:58:41.343247 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:58:41.343942 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:58:41.344481 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:58:41.344486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:58:41.344490 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:58:41.344803 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:58:41.345608 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:58:41.345617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:58:41.346111 (+ 494us) client_negotiation.cc:769] Sending connection context
0619 14:58:41.347769 (+ 1658us) client_negotiation.cc:240] Negotiation successful
0619 14:58:41.348380 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7393,"thread_start_us":7531,"threads_started":1}
I20260619 14:58:41.352044 23517 heartbeater.cc:347] Connected to a master server at 127.11.175.254:38677
I20260619 14:58:41.352401 23517 heartbeater.cc:464] Registering TS with master...
I20260619 14:58:41.353334 23517 heartbeater.cc:511] Master 127.11.175.254:38677 requested a full tablet report, sending...
I20260619 14:58:41.356256 23304 ts_manager.cc:194] Registered new tserver with Master: 9fef2aabb8454d3fa53f529e9aab9492 (127.11.175.193:42137)
I20260619 14:58:41.358424 23304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:53201
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:58:41.410406 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:42899/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38677
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 14:58:41 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881120, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
Jun 19 14:58:41 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881120, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
W20260619 14:58:41.908703 23523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:58:41.909137 23523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:58:41.909327 23523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:58:41.920884 23523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:58:41.921060 23523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:58:41.921329 23523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:58:41.935645 23523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.175.212:42899/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:58:41.937695 23523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:58:41.939473 23523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:58:41.956139 23531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:58:41.957965 23530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:58:41.961576 23533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:58:41.961900 23523 server_base.cc:1034] running on GCE node
I20260619 14:58:41.962623 23523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:58:41.969712 23523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:58:41.971108 23523 hybrid_clock.cc:648] HybridClock initialized: now 1781881121971021 us; error 82 us; skew 500 ppm
I20260619 14:58:41.971606 23523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:58:41 dist-test-slave-76rk krb5kdc[22828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881121, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:58:41.976640 23523 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:58:41.986743 23523 webserver.cc:460] Webserver started at http://127.11.175.194:35289/ using document root <none> and password file <none>
I20260619 14:58:41.987746 23523 fs_manager.cc:359] Metadata directory not provided
I20260619 14:58:41.987907 23523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:58:41.988307 23523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 14:58:41 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881121, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:58:42.015805 23523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "a364e04094ab4441b209d3c87c011fe5"
format_stamp: "Formatted at 2026-06-19 14:58:42 on dist-test-slave-76rk"
server_key: "7ef0959e621113dfda8908a11fe4e9d9"
server_key_iv: "8ffb4cd84014d2a4658d58f4dea1340e"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:42.017208 23523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "a364e04094ab4441b209d3c87c011fe5"
format_stamp: "Formatted at 2026-06-19 14:58:42 on dist-test-slave-76rk"
server_key: "7ef0959e621113dfda8908a11fe4e9d9"
server_key_iv: "8ffb4cd84014d2a4658d58f4dea1340e"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:42.034718 23523 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.010s sys 0.005s
I20260619 14:58:42.069432 23540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:58:42.071630 23523 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.006s sys 0.004s
I20260619 14:58:42.071939 23523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "a364e04094ab4441b209d3c87c011fe5"
format_stamp: "Formatted at 2026-06-19 14:58:42 on dist-test-slave-76rk"
server_key: "7ef0959e621113dfda8908a11fe4e9d9"
server_key_iv: "8ffb4cd84014d2a4658d58f4dea1340e"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:42.072286 23523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:58:42.119186 23523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:58:42.128450 23523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:58:42.128861 23523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:58:42.147193 23523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:58:42.163676 23523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:58:42.163898 23523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 14:58:42.164088 23523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:58:42.164182 23523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:58:42.263522 23523 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:34311
I20260619 14:58:42.263549 23659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:34311 every 8 connection(s)
I20260619 14:58:42.265280 23523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:58:42.271579 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 23523
I20260619 14:58:42.271831 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:58:42.272385 11967 external_mini_cluster.cc:1428] Setting key 54dabfb4483b39f5f0a3228b35cec3f3
Jun 19 14:58:42 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881121, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:58:42.297890 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:42.269626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:53049 (local address 127.11.175.254:38677)
0619 14:58:42.275099 (+ 5473us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:42.275106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:42.278832 (+ 3726us) server_negotiation.cc:407] Connection header received
0619 14:58:42.278905 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:42.278910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:42.279021 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:42.279121 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:58:42.281509 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:42.282693 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:42.285313 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:42.285961 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:42.286071 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:42.290708 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:58:42.290715 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:58:42.290720 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:58:42.290763 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:58:42.294109 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:42.295133 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:42.295138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:42.295141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:42.295282 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:42.295804 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:42.295809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:42.295812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:42.296181 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:58:42.296303 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:42.297281 (+ 978us) server_negotiation.cc:299] Negotiation successful
0619 14:58:42.297524 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5337,"thread_start_us":5229,"threads_started":1}
I20260619 14:58:42.300607 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:42.269003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38677 (local address 127.11.175.194:53049)
0619 14:58:42.275759 (+ 6756us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:42.275809 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 14:58:42.278279 (+ 2470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:58:42.279604 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:58:42.279617 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:58:42.280360 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:58:42.281270 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:42.281299 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:42.282917 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:42.282923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:42.285101 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:42.285130 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:42.286455 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:42.286463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:42.286803 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:42.287615 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:58:42.287647 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:58:42.290387 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:58:42.294296 (+ 3909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:58:42.294306 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:58:42.294327 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:58:42.294985 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:58:42.295401 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:58:42.295406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:58:42.295409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:58:42.295676 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:58:42.296379 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:58:42.296388 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:58:42.296832 (+ 444us) client_negotiation.cc:769] Sending connection context
0619 14:58:42.298511 (+ 1679us) client_negotiation.cc:240] Negotiation successful
0619 14:58:42.299127 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6434,"thread_start_us":13573,"threads_started":1}
I20260619 14:58:42.302681 23660 heartbeater.cc:347] Connected to a master server at 127.11.175.254:38677
I20260619 14:58:42.303036 23660 heartbeater.cc:464] Registering TS with master...
I20260619 14:58:42.303913 23660 heartbeater.cc:511] Master 127.11.175.254:38677 requested a full tablet report, sending...
I20260619 14:58:42.305680 23304 ts_manager.cc:194] Registered new tserver with Master: a364e04094ab4441b209d3c87c011fe5 (127.11.175.194:34311)
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
I20260619 14:58:42.306892 23304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:53049
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:58:42.350505 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:42899/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38677
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:58:42.362774 23517 heartbeater.cc:503] Master 127.11.175.254:38677 was elected leader, sending a full tablet report...
W20260619 14:58:42.767735 23666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:58:42.768030 23666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:58:42.768209 23666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:58:42.775740 23666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:58:42.775859 23666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:58:42.776084 23666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:58:42.786303 23666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.175.212:42899/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:58:42.787668 23666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:58:42.789013 23666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:58:42.802266 23671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:58:42.802233 23674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:58:42.802306 23672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:58:42.803447 23666 server_base.cc:1034] running on GCE node
I20260619 14:58:42.803990 23666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:58:42.804872 23666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:58:42.806114 23666 hybrid_clock.cc:648] HybridClock initialized: now 1781881122806072 us; error 35 us; skew 500 ppm
I20260619 14:58:42.806444 23666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:58:42 dist-test-slave-76rk krb5kdc[22828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881122, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:58:42.809705 23666 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:58:42.811201 23666 webserver.cc:460] Webserver started at http://127.11.175.195:34835/ using document root <none> and password file <none>
I20260619 14:58:42.811987 23666 fs_manager.cc:359] Metadata directory not provided
I20260619 14:58:42.812147 23666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:58:42.812558 23666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 14:58:42 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881122, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:58:42.829104 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "42fb0420c78e491f9f4ae65e6d805354"
format_stamp: "Formatted at 2026-06-19 14:58:42 on dist-test-slave-76rk"
server_key: "f462a74e125bb65d5370aa777e437912"
server_key_iv: "a751af4a2eaddf3eca9458430f40e9d0"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:42.829984 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "42fb0420c78e491f9f4ae65e6d805354"
format_stamp: "Formatted at 2026-06-19 14:58:42 on dist-test-slave-76rk"
server_key: "f462a74e125bb65d5370aa777e437912"
server_key_iv: "a751af4a2eaddf3eca9458430f40e9d0"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:42.837133 23666 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260619 14:58:42.855171 23681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:58:42.857101 23666 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 14:58:42.857316 23666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "42fb0420c78e491f9f4ae65e6d805354"
format_stamp: "Formatted at 2026-06-19 14:58:42 on dist-test-slave-76rk"
server_key: "f462a74e125bb65d5370aa777e437912"
server_key_iv: "a751af4a2eaddf3eca9458430f40e9d0"
server_key_version: "kuduclusterkey@0"
I20260619 14:58:42.857618 23666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:58:42.877964 23666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:58:42.882200 23666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:58:42.882513 23666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:58:42.883648 23666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:58:42.885054 23666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:58:42.885126 23666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:58:42.885257 23666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:58:42.885315 23666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:58:42.921743 23666 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:44779
I20260619 14:58:42.921766 23793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:44779 every 8 connection(s)
I20260619 14:58:42.923192 23666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:58:42.924126 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 23666
I20260619 14:58:42.924317 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:58:42.924870 11967 external_mini_cluster.cc:1428] Setting key de488d6438719c77795a805d54695338
Jun 19 14:58:42 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881122, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:58:42.945571 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:42.926402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:60471 (local address 127.11.175.254:38677)
0619 14:58:42.926992 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:42.926999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:42.928312 (+ 1313us) server_negotiation.cc:407] Connection header received
0619 14:58:42.930495 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:42.930499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:42.930576 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:42.930797 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:58:42.932272 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:42.933043 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:42.934808 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:42.935244 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:42.935346 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:42.938785 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:58:42.938790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:58:42.938793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:58:42.938827 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:58:42.942306 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:42.943204 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:42.943208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:42.943211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:42.943348 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:42.943790 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:42.943794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:42.943797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:42.944078 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:58:42.944232 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:42.944939 (+ 707us) server_negotiation.cc:299] Negotiation successful
0619 14:58:42.945169 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":300,"threads_started":1}
I20260619 14:58:42.946617 23797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:42.927211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38677 (local address 127.11.175.195:60471)
0619 14:58:42.928147 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:42.928198 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 14:58:42.930177 (+ 1979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:58:42.930985 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:58:42.930993 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:58:42.931440 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:58:42.932088 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:42.932107 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:42.933188 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:42.933192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:42.934507 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:42.934605 (+ 98us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:42.935484 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:42.935488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:42.935706 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:42.936439 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:58:42.936463 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:58:42.938547 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:58:42.942469 (+ 3922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:58:42.942479 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:58:42.942499 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:58:42.943070 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:58:42.943466 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:58:42.943472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:58:42.943475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:58:42.943668 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:58:42.944202 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:58:42.944210 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:58:42.944575 (+ 365us) client_negotiation.cc:769] Sending connection context
0619 14:58:42.944846 (+ 271us) client_negotiation.cc:240] Negotiation successful
0619 14:58:42.945410 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":389,"threads_started":1}
I20260619 14:58:42.948499 23794 heartbeater.cc:347] Connected to a master server at 127.11.175.254:38677
I20260619 14:58:42.948850 23794 heartbeater.cc:464] Registering TS with master...
I20260619 14:58:42.949713 23794 heartbeater.cc:511] Master 127.11.175.254:38677 requested a full tablet report, sending...
I20260619 14:58:42.951335 23304 ts_manager.cc:194] Registered new tserver with Master: 42fb0420c78e491f9f4ae65e6d805354 (127.11.175.195:44779)
I20260619 14:58:42.952270 23304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:60471
I20260619 14:58:42.954370 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 14:58:42.974392 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:42.959965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58768 (local address 127.11.175.254:38677)
0619 14:58:42.960252 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:42.960258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:42.960623 (+ 365us) server_negotiation.cc:407] Connection header received
0619 14:58:42.960835 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:42.960840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:42.960929 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:42.961061 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:58:42.962725 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:42.963625 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:42.965034 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:42.965464 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:42.965606 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:42.968320 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:58:42.968326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:58:42.968329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:58:42.968368 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:58:42.971504 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:42.972268 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:42.972276 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:42.972279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:42.972424 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:42.972810 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:42.972816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:42.972819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:42.973118 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:58:42.973421 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:42.973785 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 14:58:42.974035 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260619 14:58:42.979565 23304 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:58:42.987341 23304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:58:43.011520 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.000710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.11.175.195:44779)
0619 14:58:43.001448 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:43.001460 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:43.001978 (+ 518us) server_negotiation.cc:407] Connection header received
0619 14:58:43.002360 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:43.002368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:43.002627 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:43.002816 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:58:43.003883 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.005516 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.008352 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.009809 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.009975 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.010157 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:43.010936 (+ 779us) server_negotiation.cc:299] Negotiation successful
0619 14:58:43.011123 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":359,"threads_started":1}
I20260619 14:58:43.012461 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.000296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:44779 (local address 127.0.0.1:56032)
0619 14:58:43.001825 (+ 1529us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.001866 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 14:58:43.002117 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:58:43.003216 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:58:43.003223 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:58:43.003252 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:58:43.003666 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:43.003680 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.005706 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.005713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:43.008172 (+ 2459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:43.008185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.010354 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.010360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:43.010575 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.010624 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 14:58:43.011812 (+ 1188us) client_negotiation.cc:240] Negotiation successful
0619 14:58:43.012032 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1290,"spinlock_wait_cycles":5376,"thread_start_us":1149,"threads_started":1}
I20260619 14:58:43.016443 23810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.000364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:42137 (local address 127.0.0.1:44606)
0619 14:58:43.001920 (+ 1556us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.001938 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 14:58:43.002114 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:58:43.004187 (+ 2073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:58:43.004191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:58:43.004217 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:58:43.004640 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:43.004652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.007822 (+ 3170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.007827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:43.011093 (+ 3266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:43.011107 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.015607 (+ 4500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.015615 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:43.015790 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.015822 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 14:58:43.015923 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 14:58:43.016087 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1440,"spinlock_wait_cycles":896,"thread_start_us":1298,"threads_started":1}
I20260619 14:58:43.016547 23809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.000744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.11.175.193:42137)
0619 14:58:43.003319 (+ 2575us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:43.003329 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:43.003359 (+ 30us) server_negotiation.cc:407] Connection header received
0619 14:58:43.003472 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:43.003479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:43.003711 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:43.003895 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:58:43.004848 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.007660 (+ 2812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.011269 (+ 3609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.015175 (+ 3906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.015400 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.015590 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:43.015998 (+ 408us) server_negotiation.cc:299] Negotiation successful
0619 14:58:43.016188 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2414,"thread_start_us":2277,"threads_started":1}
I20260619 14:58:43.017014 23729 tablet_service.cc:1467] Processing CreateTablet for tablet d4eaedd9da5b42649bac42b5b316ee77 (DEFAULT_TABLE table=test-table [id=88f785b452184c7499dfdb5801f068de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:58:43.018548 23729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4eaedd9da5b42649bac42b5b316ee77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:58:43.020087 23452 tablet_service.cc:1467] Processing CreateTablet for tablet d4eaedd9da5b42649bac42b5b316ee77 (DEFAULT_TABLE table=test-table [id=88f785b452184c7499dfdb5801f068de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:58:43.021273 23452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4eaedd9da5b42649bac42b5b316ee77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:58:43.027669 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.002278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:34311 (local address 127.0.0.1:54928)
0619 14:58:43.013982 (+ 11704us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.014003 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:58:43.014103 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:58:43.015307 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:58:43.015311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:58:43.015339 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:58:43.015525 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:43.015540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.022895 (+ 7355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.022902 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:43.024889 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:43.024905 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.026804 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.026811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:43.027010 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.027047 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 14:58:43.027144 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 14:58:43.027292 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11581,"thread_start_us":12545,"threads_started":1}
I20260619 14:58:43.028057 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.002677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54928 (local address 127.11.175.194:34311)
0619 14:58:43.014665 (+ 11988us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:43.014675 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:43.014706 (+ 31us) server_negotiation.cc:407] Connection header received
0619 14:58:43.014809 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:43.014817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:43.015101 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:43.015296 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:58:43.015700 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.022704 (+ 7004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.025112 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.026594 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.026817 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.027001 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:43.027204 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 14:58:43.027437 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11789,"thread_start_us":11664,"threads_started":1}
I20260619 14:58:43.031670 23813 tablet_bootstrap.cc:492] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: Bootstrap starting.
I20260619 14:58:43.032012 23595 tablet_service.cc:1467] Processing CreateTablet for tablet d4eaedd9da5b42649bac42b5b316ee77 (DEFAULT_TABLE table=test-table [id=88f785b452184c7499dfdb5801f068de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:58:43.033571 23595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4eaedd9da5b42649bac42b5b316ee77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:58:43.034610 23814 tablet_bootstrap.cc:492] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492: Bootstrap starting.
I20260619 14:58:43.036178 23813 tablet_bootstrap.cc:654] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: Neither blocks nor log segments found. Creating new log.
I20260619 14:58:43.037511 23813 log.cc:824] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: Log is configured to *not* fsync() on all Append() calls
I20260619 14:58:43.038466 23814 tablet_bootstrap.cc:654] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492: Neither blocks nor log segments found. Creating new log.
I20260619 14:58:43.039742 23814 log.cc:824] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492: Log is configured to *not* fsync() on all Append() calls
I20260619 14:58:43.040876 23813 tablet_bootstrap.cc:492] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: No bootstrap required, opened a new log
I20260619 14:58:43.041208 23813 ts_tablet_manager.cc:1397] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s
I20260619 14:58:43.043018 23814 tablet_bootstrap.cc:492] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492: No bootstrap required, opened a new log
I20260619 14:58:43.043293 23814 ts_tablet_manager.cc:1397] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260619 14:58:43.046476 23817 tablet_bootstrap.cc:492] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: Bootstrap starting.
I20260619 14:58:43.049885 23817 tablet_bootstrap.cc:654] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: Neither blocks nor log segments found. Creating new log.
I20260619 14:58:43.051101 23817 log.cc:824] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: Log is configured to *not* fsync() on all Append() calls
I20260619 14:58:43.052757 23813 raft_consensus.cc:348] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fb0420c78e491f9f4ae65e6d805354" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44779 } } peers { permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 } } peers { permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 } }
I20260619 14:58:43.053265 23813 raft_consensus.cc:374] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:58:43.053385 23813 raft_consensus.cc:729] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fb0420c78e491f9f4ae65e6d805354, State: Initialized, Role: FOLLOWER
I20260619 14:58:43.054222 23813 consensus_queue.cc:260] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fb0420c78e491f9f4ae65e6d805354" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44779 } } peers { permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 } } peers { permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 } }
I20260619 14:58:43.054550 23814 raft_consensus.cc:348] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fb0420c78e491f9f4ae65e6d805354" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44779 } } peers { permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 } } peers { permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 } }
I20260619 14:58:43.054953 23817 tablet_bootstrap.cc:492] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: No bootstrap required, opened a new log
I20260619 14:58:43.055099 23814 raft_consensus.cc:374] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:58:43.055220 23814 raft_consensus.cc:729] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fef2aabb8454d3fa53f529e9aab9492, State: Initialized, Role: FOLLOWER
I20260619 14:58:43.055274 23817 ts_tablet_manager.cc:1397] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260619 14:58:43.055706 23794 heartbeater.cc:503] Master 127.11.175.254:38677 was elected leader, sending a full tablet report...
I20260619 14:58:43.055927 23814 consensus_queue.cc:260] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fb0420c78e491f9f4ae65e6d805354" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44779 } } peers { permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 } } peers { permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 } }
I20260619 14:58:43.056311 23813 ts_tablet_manager.cc:1428] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: Time spent starting tablet: real 0.015s user 0.016s sys 0.001s
I20260619 14:58:43.058074 23814 ts_tablet_manager.cc:1428] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260619 14:58:43.065435 23817 raft_consensus.cc:348] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fb0420c78e491f9f4ae65e6d805354" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44779 } } peers { permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 } } peers { permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 } }
I20260619 14:58:43.065919 23817 raft_consensus.cc:374] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:58:43.066047 23817 raft_consensus.cc:729] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a364e04094ab4441b209d3c87c011fe5, State: Initialized, Role: FOLLOWER
I20260619 14:58:43.066779 23817 consensus_queue.cc:260] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fb0420c78e491f9f4ae65e6d805354" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44779 } } peers { permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 } } peers { permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 } }
I20260619 14:58:43.067873 23660 heartbeater.cc:503] Master 127.11.175.254:38677 was elected leader, sending a full tablet report...
I20260619 14:58:43.068593 23817 ts_tablet_manager.cc:1428] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
W20260619 14:58:43.070407 23518 tablet.cc:2369] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:58:43.135599 23819 raft_consensus.cc:482] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:58:43.135860 23819 raft_consensus.cc:504] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fb0420c78e491f9f4ae65e6d805354" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44779 } } peers { permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 } } peers { permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 } }
I20260619 14:58:43.137507 23819 leader_election.cc:288] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fef2aabb8454d3fa53f529e9aab9492 (127.11.175.193:42137), a364e04094ab4441b209d3c87c011fe5 (127.11.175.194:34311)
I20260619 14:58:43.143862 23809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.137891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:45823 (local address 127.11.175.193:42137)
0619 14:58:43.138111 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:43.138119 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:43.138143 (+ 24us) server_negotiation.cc:407] Connection header received
0619 14:58:43.138223 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:43.138229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:43.138306 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:43.138412 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:58:43.139005 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.140032 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.141899 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.143072 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.143187 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.143238 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:43.143584 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 14:58:43.143673 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 14:58:43.143971 23797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.137756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:42137 (local address 127.11.175.195:45823)
0619 14:58:43.137928 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.137948 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:58:43.138099 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:58:43.138569 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:58:43.138578 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:58:43.138605 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:58:43.138823 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:43.138831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.140160 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.140166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:43.141738 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:43.141749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.143205 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.143211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:43.143397 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.143433 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 14:58:43.143554 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 14:58:43.143691 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":1}
I20260619 14:58:43.144855 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.138265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:42165 (local address 127.11.175.194:34311)
0619 14:58:43.138458 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:43.138465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:43.138932 (+ 467us) server_negotiation.cc:407] Connection header received
0619 14:58:43.139066 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:43.139069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:43.139135 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:43.139203 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:58:43.139770 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.140754 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.142825 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.143853 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.144022 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.144097 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:43.144585 (+ 488us) server_negotiation.cc:299] Negotiation successful
0619 14:58:43.144682 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 14:58:43.145020 23822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.138110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:34311 (local address 127.11.175.195:42165)
0619 14:58:43.138822 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.138836 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 14:58:43.138957 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:58:43.139244 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:58:43.139250 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:58:43.139275 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:58:43.139633 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:43.139644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.140898 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.140901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:43.142670 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:58:43.142684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.144281 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.144291 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:58:43.144409 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.144430 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 14:58:43.144527 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 14:58:43.144659 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":21,"thread_start_us":401,"threads_started":1}
I20260619 14:58:43.145079 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4eaedd9da5b42649bac42b5b316ee77" candidate_uuid: "42fb0420c78e491f9f4ae65e6d805354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef2aabb8454d3fa53f529e9aab9492" is_pre_election: true
I20260619 14:58:43.145627 23472 raft_consensus.cc:2454] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42fb0420c78e491f9f4ae65e6d805354 in term 0.
I20260619 14:58:43.145925 23615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4eaedd9da5b42649bac42b5b316ee77" candidate_uuid: "42fb0420c78e491f9f4ae65e6d805354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a364e04094ab4441b209d3c87c011fe5" is_pre_election: true
I20260619 14:58:43.146297 23684 leader_election.cc:302] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42fb0420c78e491f9f4ae65e6d805354, 9fef2aabb8454d3fa53f529e9aab9492; no voters:
I20260619 14:58:43.146437 23615 raft_consensus.cc:2454] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42fb0420c78e491f9f4ae65e6d805354 in term 0.
I20260619 14:58:43.146771 23819 raft_consensus.cc:2781] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:58:43.146929 23819 raft_consensus.cc:482] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:58:43.147070 23819 raft_consensus.cc:3037] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:58:43.149281 23819 raft_consensus.cc:504] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fb0420c78e491f9f4ae65e6d805354" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44779 } } peers { permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 } } peers { permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 } }
I20260619 14:58:43.149837 23819 leader_election.cc:288] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [CANDIDATE]: Term 1 election: Requested vote from peers 9fef2aabb8454d3fa53f529e9aab9492 (127.11.175.193:42137), a364e04094ab4441b209d3c87c011fe5 (127.11.175.194:34311)
I20260619 14:58:43.150276 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4eaedd9da5b42649bac42b5b316ee77" candidate_uuid: "42fb0420c78e491f9f4ae65e6d805354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fef2aabb8454d3fa53f529e9aab9492"
I20260619 14:58:43.150396 23615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4eaedd9da5b42649bac42b5b316ee77" candidate_uuid: "42fb0420c78e491f9f4ae65e6d805354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a364e04094ab4441b209d3c87c011fe5"
I20260619 14:58:43.150484 23472 raft_consensus.cc:3037] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:58:43.150672 23615 raft_consensus.cc:3037] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:58:43.152697 23472 raft_consensus.cc:2454] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42fb0420c78e491f9f4ae65e6d805354 in term 1.
I20260619 14:58:43.152912 23615 raft_consensus.cc:2454] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42fb0420c78e491f9f4ae65e6d805354 in term 1.
I20260619 14:58:43.153100 23684 leader_election.cc:302] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42fb0420c78e491f9f4ae65e6d805354, 9fef2aabb8454d3fa53f529e9aab9492; no voters:
I20260619 14:58:43.153391 23819 raft_consensus.cc:2781] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:58:43.153824 23819 raft_consensus.cc:686] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 1 LEADER]: Becoming Leader. State: Replica: 42fb0420c78e491f9f4ae65e6d805354, State: Running, Role: LEADER
I20260619 14:58:43.154301 23819 consensus_queue.cc:237] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fb0420c78e491f9f4ae65e6d805354" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44779 } } peers { permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 } } peers { permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 } }
I20260619 14:58:43.158890 23304 catalog_manager.cc:5495] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42fb0420c78e491f9f4ae65e6d805354 (127.11.175.195). New cstate: current_term: 1 leader_uuid: "42fb0420c78e491f9f4ae65e6d805354" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fb0420c78e491f9f4ae65e6d805354" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 } health_report { overall_health: UNKNOWN } } }
W20260619 14:58:43.175034 23795 tablet.cc:2369] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:58:43.207763 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.202392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.11.175.195:44779)
0619 14:58:43.202606 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:43.202611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:43.202629 (+ 18us) server_negotiation.cc:407] Connection header received
0619 14:58:43.202827 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:43.202831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:43.202918 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:43.203051 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:58:43.203406 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.204123 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.205529 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.205918 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.206010 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.206542 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:58:43.207231 (+ 689us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:58:43.207341 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:43.207460 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 14:58:43.207529 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 14:58:43.215689 23472 raft_consensus.cc:1261] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492 [term 1 FOLLOWER]: Refusing update from remote peer 42fb0420c78e491f9f4ae65e6d805354: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:58:43.215716 23615 raft_consensus.cc:1261] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5 [term 1 FOLLOWER]: Refusing update from remote peer 42fb0420c78e491f9f4ae65e6d805354: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:58:43.216635 23819 consensus_queue.cc:1035] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fef2aabb8454d3fa53f529e9aab9492" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 42137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:58:43.216974 23823 consensus_queue.cc:1035] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a364e04094ab4441b209d3c87c011fe5" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:58:43.228857 23828 mvcc.cc:205] Tried to move back new op lower bound from 7298585080681771008 to 7298585080444030976. Current Snapshot: MvccSnapshot[applied={T|T < 7298585080681771008}]
I20260619 14:58:43.245770 23304 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:58:43.246068 23304 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:58:43.250048 23304 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 374a619379d64f92991cf301716902ad: Sending DeleteTablet for 3 replicas of tablet d4eaedd9da5b42649bac42b5b316ee77
I20260619 14:58:43.251389 23729 tablet_service.cc:1514] Processing DeleteTablet for tablet d4eaedd9da5b42649bac42b5b316ee77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:58:43 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:56032
I20260619 14:58:43.251830 23452 tablet_service.cc:1514] Processing DeleteTablet for tablet d4eaedd9da5b42649bac42b5b316ee77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:58:43 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:44606
I20260619 14:58:43.252170 23595 tablet_service.cc:1514] Processing DeleteTablet for tablet d4eaedd9da5b42649bac42b5b316ee77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:58:43 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:54928
I20260619 14:58:43.254701 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 23384
I20260619 14:58:43.262804 23838 tablet_replica.cc:330] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: stopping tablet replica
I20260619 14:58:43.278816 23838 raft_consensus.cc:2229] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:58:43.279402 23838 raft_consensus.cc:2258] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5 [term 1 FOLLOWER]: Raft consensus is shut down!
Jun 19 14:58:43 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:58:43.283000 23838 ts_tablet_manager.cc:1905] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:58:43.287781 23837 tablet_replica.cc:330] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492: stopping tablet replica
I20260619 14:58:43.287920 23836 tablet_replica.cc:330] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: stopping tablet replica
I20260619 14:58:43.288393 23836 raft_consensus.cc:2229] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:58:43.288396 23837 raft_consensus.cc:2229] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:58:43.289131 23836 raft_consensus.cc:2258] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:58:43.289247 23837 raft_consensus.cc:2258] T d4eaedd9da5b42649bac42b5b316ee77 P 9fef2aabb8454d3fa53f529e9aab9492 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:58:43.290576 23809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.256921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44622 (local address 127.11.175.193:42137)
0619 14:58:43.257155 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:43.257164 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:43.259818 (+ 2654us) server_negotiation.cc:407] Connection header received
0619 14:58:43.259996 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:43.260004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:43.260125 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:43.260245 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:58:43.265563 (+ 5318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.267236 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.270185 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.271209 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.271346 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.281552 (+ 10206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:58:43.281573 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:58:43.281592 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:58:43.281648 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:58:43.285945 (+ 4297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:43.286999 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:43.287009 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:43.287013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:43.287194 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:43.287813 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:43.287818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:43.287821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:43.288526 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:58:43.288955 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:43.289755 (+ 800us) server_negotiation.cc:299] Negotiation successful
0619 14:58:43.290037 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 14:58:43.291769 23836 ts_tablet_manager.cc:1905] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:58:43.292761 23512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:58:43.300544 23836 ts_tablet_manager.cc:1918] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:58:43.300746 23836 log.cc:1196] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/d4eaedd9da5b42649bac42b5b316ee77
I20260619 14:58:43.301292 23836 ts_tablet_manager.cc:1939] T d4eaedd9da5b42649bac42b5b316ee77 P 42fb0420c78e491f9f4ae65e6d805354: Deleting consensus metadata
I20260619 14:58:43.302722 23838 ts_tablet_manager.cc:1918] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:58:43.302904 23838 log.cc:1196] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/d4eaedd9da5b42649bac42b5b316ee77
I20260619 14:58:43.303452 23838 ts_tablet_manager.cc:1939] T d4eaedd9da5b42649bac42b5b316ee77 P a364e04094ab4441b209d3c87c011fe5: Deleting consensus metadata
I20260619 14:58:43.303544 23289 catalog_manager.cc:4841] TS 42fb0420c78e491f9f4ae65e6d805354 (127.11.175.195:44779): tablet d4eaedd9da5b42649bac42b5b316ee77 (table test-table [id=88f785b452184c7499dfdb5801f068de]) successfully deleted
I20260619 14:58:43.305342 23290 catalog_manager.cc:4841] TS a364e04094ab4441b209d3c87c011fe5 (127.11.175.194:34311): tablet d4eaedd9da5b42649bac42b5b316ee77 (table test-table [id=88f785b452184c7499dfdb5801f068de]) successfully deleted
I20260619 14:58:43.395620 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 23384
I20260619 14:58:43.413069 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 23523
W20260619 14:58:43.413069 23290 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.175.193:42137 (error 108)
I20260619 14:58:43.415489 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.414942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:42137 (local address 127.0.0.1:44626)
0619 14:58:43.415159 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.415275 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:42137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260619 14:58:43.415750 23290 catalog_manager.cc:4584] TS 9fef2aabb8454d3fa53f529e9aab9492 (127.11.175.193:42137): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4eaedd9da5b42649bac42b5b316ee77: Network error: Client connection negotiation failed: client connection to 127.11.175.193:42137: connect: Connection refused (error 111)
Jun 19 14:58:43 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:58:43.431496 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.414303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54944 (local address 127.11.175.194:34311)
0619 14:58:43.414576 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:43.414583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:43.414605 (+ 22us) server_negotiation.cc:407] Connection header received
0619 14:58:43.414693 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:43.414699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:43.414781 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:43.414896 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:58:43.416494 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.417677 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.419247 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.419885 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.420021 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.424442 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:58:43.424461 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:58:43.424481 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:58:43.424538 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:58:43.428391 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:43.429210 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:43.429221 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:43.429225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:43.429392 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:43.429772 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:43.429777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:43.429782 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:43.430265 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:58:43.430583 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:43.430857 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 14:58:43.431089 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 14:58:43.432240 23655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:58:43.469444 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.468904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:42137 (local address 127.0.0.1:44640)
0619 14:58:43.469163 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.469271 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:42137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260619 14:58:43.470705 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.470211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:42137 (local address 127.0.0.1:44654)
0619 14:58:43.470379 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.470500 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:42137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 14:58:43.523782 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 23523
I20260619 14:58:43.531533 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.530933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:42137 (local address 127.0.0.1:44664)
0619 14:58:43.531226 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.531330 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:42137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260619 14:58:43.532806 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.532329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:42137 (local address 127.0.0.1:44676)
0619 14:58:43.532527 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.532638 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:42137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 14:58:43.542316 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 23666
Jun 19 14:58:43 dist-test-slave-76rk krb5kdc[22828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:58:43.562485 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.543317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56044 (local address 127.11.175.195:44779)
0619 14:58:43.543514 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 14:58:43.543519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:58:43.543537 (+ 18us) server_negotiation.cc:407] Connection header received
0619 14:58:43.543643 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:58:43.543647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:58:43.543708 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:58:43.543790 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:58:43.545590 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.546368 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.548648 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:58:43.549110 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:58:43.549210 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:58:43.555227 (+ 6017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:58:43.555245 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:58:43.555272 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:58:43.555338 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:58:43.559573 (+ 4235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:43.560284 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:43.560295 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:43.560299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:43.560510 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:58:43.560887 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:58:43.560892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:58:43.560895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:58:43.561342 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:58:43.561684 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0619 14:58:43.561850 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 14:58:43.562085 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 14:58:43.563359 23789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:58:43.613730 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.613168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:42137 (local address 127.0.0.1:44678)
0619 14:58:43.613418 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.613538 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:42137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260619 14:58:43.615015 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.614464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:42137 (local address 127.0.0.1:44694)
0619 14:58:43.614728 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.614827 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:42137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260619 14:58:43.688616 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 23666
I20260619 14:58:43.708348 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 23272
I20260619 14:58:43.709589 23334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:58:43.820673 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:58:43.819782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:42137 (local address 127.0.0.1:44706)
0619 14:58:43.820234 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0619 14:58:43.820417 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:42137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294}
I20260619 14:58:43.820879 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 23272
I20260619 14:58:43.843236 11967 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 14:58:44.178969 11967 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 14:58:44.179271 11967 mini_ranger.cc:66] Stopping Ranger...
I20260619 14:58:44.525103 11967 mini_ranger.cc:68] Stopped Ranger
2026-06-19 14:58:44.525 UTC [22858] LOG: received smart shutdown request
2026-06-19 14:58:44.528 UTC [22858] LOG: background worker "logical replication launcher" (PID 22867) exited with exit code 1
2026-06-19 14:58:44.528 UTC [22862] LOG: shutting down
2026-06-19 14:58:44.698 UTC [22858] LOG: database system is shut down
2026-06-19T14:58:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52568 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:58:44 dist-test-slave-76rk krb5kdc[23853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:58:44 dist-test-slave-76rk krb5kdc[23853](info): set up 2 sockets
Jun 19 14:58:44 dist-test-slave-76rk krb5kdc[23853](info): commencing operation
krb5kdc: starting...
W20260619 14:58:46.015372 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:58:46 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:58:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:58:46Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.175.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.175.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/rangeradmin_127.11.175.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/rangeradmin_127.11.175.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.175.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.175.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/rangerlookup_127.11.175.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/rangerlookup_127.11.175.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.175.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.175.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/HTTP_127.11.175.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/HTTP_127.11.175.212@KRBTEST.COM.keytab.
I20260619 14:58:46.359081 11967 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDoOfXv/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/postgres -l logfile start
2026-06-19 14:58:48.864 UTC [23883] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 14:58:48.864 UTC [23883] LOG: listening on IPv4 address "127.11.175.212", port 38929
2026-06-19 14:58:48.871 UTC [23883] LOG: listening on Unix socket "/tmp/.s.PGSQL.38929"
2026-06-19 14:58:48.894 UTC [23886] LOG: database system was shut down at 2026-06-19 14:58:47 UTC
2026-06-19 14:58:48.898 UTC [23883] LOG: database system is ready to accept connections
I20260619 14:58:49.894522 11967 mini_postgres.cc:96] Postgres bound to 38929
2026-06-19 14:58:49.914 UTC [23894] FATAL: database "slave" does not exist
127.11.175.212:38929 - accepting connections
I20260619 14:58:49.915488 11967 mini_ranger.cc:161] Starting Ranger...
I20260619 14:58:49.943681 11967 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 14:58:50.479430 11967 mini_ranger.cc:87] Created ranger Postgres database
I20260619 14:58:50.479552 11967 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-admin
2026-06-19 14:58:50,794 [I] DB FLAVOR :POSTGRES
2026-06-19 14:58:50,795 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 14:58:50,795 [I] Checking connection..
2026-06-19 14:58:50,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 14:58:51,050 [I] Checking connection passed.
2026-06-19 14:58:51,051 [I] --------- Verifying version history table ---------
2026-06-19 14:58:51,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 14:58:51,315 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 14:58:51,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 14:58:51,567 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 14:58:51,567 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 14:58:51,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 14:58:51.827 UTC [23958] WARNING: there is no transaction in progress
2026-06-19 14:58:51,833 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 14:58:51,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 14:58:52,083 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 14:58:52,083 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 14:58:52,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 14:58:52,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 14:58:52,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-76rk.c.gcp-upstream.internal','N') ;"
2026-06-19 14:58:52,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 14:58:53,118 [I] Table x_portal_user does not exist in database ranger
2026-06-19 14:58:53,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 14:58:53,361 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 14:58:53,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 14:58:53,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 14:58:53,888 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 14:58:53,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 14:58:54.844 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:54.862 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:54.880 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:54.896 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:54.912 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:54.930 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:54.987 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:54.994 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:55.008 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:55.019 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:55.030 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:55.043 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:55.053 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:55.059 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:55.065 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:55.072 UTC [24066] WARNING: there is no transaction in progress
2026-06-19 14:58:55,698 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 14:58:55,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 14:58:55,943 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 14:58:55,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 14:58:56,197 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 14:58:56,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 14:58:56,461 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 14:58:56,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 14:58:56,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 14:58:56,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-76rk.c.gcp-upstream.internal';"
2026-06-19 14:58:57,217 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 14:58:57,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 14:58:57,480 [I] DB_PATCHES have already been applied
I20260619 14:58:57.486851 11967 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDoOfXv/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 14:58:57.486936 11967 mini_ranger.cc:193] Using host: 127.11.175.212
I20260619 14:58:57.503202 11967 mini_ranger.cc:227] Ranger admin URL: http://127.11.175.212:33907
Jun 19, 2026 2:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:58:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:58:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:58:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 14:58:58.683460 11967 mini_ranger.cc:160] Time spent starting Ranger: real 8.768s user 0.005s sys 0.039s
Jun 19, 2026 2:59:18 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:59:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 2:59:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:59:18 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:59:19 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 14:59:20.470624 11967 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.175.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.175.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/rangerkms_127.11.175.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.175.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/rangerkms_127.11.175.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.175.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.175.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.175.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/HTTP_127.11.175.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.175.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/HTTP_127.11.175.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 14:59:20 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881160, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 14:59:20.666445 11967 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 14:59:20.689168 11967 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 14:59:21.526243 11967 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 14:59:21.526364 11967 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-kms
I20260619 14:59:21.526424 11967 mini_ranger_kms.cc:227] Using postgres at 127.11.175.212:38929
2026-06-19 14:59:21,948 [I] DB FLAVOR :POSTGRES
2026-06-19 14:59:21,949 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 14:59:21,949 [I] Checking connection
2026-06-19 14:59:21,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 14:59:22,187 [I] connection success
2026-06-19 14:59:22,187 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 14:59:22,187 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 14:59:22,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 14:59:22,443 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 14:59:22,444 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 14:59:22,444 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 14:59:22,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.175.212:38929/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 14:59:22,714 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 14:59:23.223232 11967 mini_ranger_kms.cc:310] Created kms service
I20260619 14:59:23.450212 11967 mini_ranger_kms.cc:326] Created kudu user
I20260619 14:59:23.496203 11967 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 14:59:23 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881160, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:59:23.924266 11967 mini_ranger_kms.cc:384] Added ranger policy
I20260619 14:59:23.924611 11967 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDoOfXv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDoOfXv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 14:59:23.924696 11967 mini_ranger_kms.cc:240] Using host: 127.11.175.212
I20260619 14:59:23.945333 11967 mini_ranger_kms.cc:276] Ranger KMS PID: 24246
I20260619 14:59:23.945608 11967 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.175.212:40623
Jun 19, 2026 2:59:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:59:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:59:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:59:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:59:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:59:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 14:59:26.368798 11967 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.702s user 0.088s sys 0.354s
I20260619 14:59:26.369046 11967 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 14:59:26.369233 11967 mini_ranger_kms.cc:401] 127.11.175.212:40623/kms/v1/keys
Jun 19 14:59:26 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781881166, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.175.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 14:59:26 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881166, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.175.212@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
Jun 19 14:59:27 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881166, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.175.212@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
Jun 19 14:59:29 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:59:29.710229 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:44305
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:44305
--ranger_config_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:40623/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:59:30.077273 24297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:30.077595 24297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:30.077778 24297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:30.085491 24297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:59:30.085628 24297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:30.085785 24297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:30.085937 24297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:59:30.086108 24297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:59:30.097723 24297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.175.212:40623/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:44305
--ranger_config_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:44305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:30.099282 24297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:30.100665 24297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:30.113854 24303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:30.113981 24302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:30.114007 24305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:30.114730 24297 server_base.cc:1034] running on GCE node
I20260619 14:59:30.115612 24297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:30.117203 24297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:30.118427 24297 hybrid_clock.cc:648] HybridClock initialized: now 1781881170118386 us; error 40 us; skew 500 ppm
I20260619 14:59:30.118847 24297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:30 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881170, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:30.126729 24297 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:59:30.128463 24297 webserver.cc:460] Webserver started at http://127.11.175.254:41269/ using document root <none> and password file <none>
I20260619 14:59:30.129170 24297 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:30.129320 24297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:30.129637 24297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 14:59:30 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881170, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:59:30.194053 24297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "460da7dff1754e7e8c5a18c885ef92df"
format_stamp: "Formatted at 2026-06-19 14:59:30 on dist-test-slave-76rk"
server_key: "d8077ca3b09095ee4e2873ff05aa63bd"
server_key_iv: "61782ebd72628d265657baaafa872d2d"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:30.195358 24297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "460da7dff1754e7e8c5a18c885ef92df"
format_stamp: "Formatted at 2026-06-19 14:59:30 on dist-test-slave-76rk"
server_key: "d8077ca3b09095ee4e2873ff05aa63bd"
server_key_iv: "61782ebd72628d265657baaafa872d2d"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:30.203689 24297 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260619 14:59:30.223906 24312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:30.225692 24297 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 14:59:30.225895 24297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "460da7dff1754e7e8c5a18c885ef92df"
format_stamp: "Formatted at 2026-06-19 14:59:30 on dist-test-slave-76rk"
server_key: "d8077ca3b09095ee4e2873ff05aa63bd"
server_key_iv: "61782ebd72628d265657baaafa872d2d"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:30.226159 24297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:30.256420 24297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:30.260572 24297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:30.260839 24297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:30.280957 24297 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:44305
I20260619 14:59:30.280992 24363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:44305 every 8 connection(s)
I20260619 14:59:30.282441 24297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:59:30.286505 24364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:30.292614 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 24297
I20260619 14:59:30.292877 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:59:30.293736 11967 external_mini_cluster.cc:1428] Setting key f22d56899ababfc4640259d52f804997
I20260619 14:59:30.298702 24364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: Bootstrap starting.
I20260619 14:59:30.302727 24364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:30.304050 24364 log.cc:824] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:30.307844 24364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: No bootstrap required, opened a new log
Jun 19 14:59:30 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:30.318495 24364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } }
I20260619 14:59:30.319092 24364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:30.319217 24364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460da7dff1754e7e8c5a18c885ef92df, State: Initialized, Role: FOLLOWER
I20260619 14:59:30.320006 24364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } }
I20260619 14:59:30.320297 24364 raft_consensus.cc:388] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:59:30.320441 24364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:59:30.320627 24364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:30.320859 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:30.296191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37778 (local address 127.11.175.254:44305)
0619 14:59:30.297076 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:30.297087 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:30.297129 (+ 42us) server_negotiation.cc:407] Connection header received
0619 14:59:30.299049 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:30.299073 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:30.299499 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:30.299838 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:30.301704 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:30.303337 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:30.305259 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:30.306193 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:30.306352 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:30.311334 (+ 4982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:30.311346 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:30.311373 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:30.311431 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:30.315704 (+ 4273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:30.316550 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:30.316561 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:30.316571 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:30.316804 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:30.317277 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:30.317282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:30.317286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:30.318086 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:30.318369 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:30.318925 (+ 556us) server_negotiation.cc:299] Negotiation successful
0619 14:59:30.319394 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":439,"threads_started":1}
I20260619 14:59:30.323195 24364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } }
I20260619 14:59:30.323750 24364 leader_election.cc:302] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 460da7dff1754e7e8c5a18c885ef92df; no voters:
I20260619 14:59:30.324368 24364 leader_election.cc:288] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:59:30.324514 24369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:59:30.325289 24369 raft_consensus.cc:686] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 1 LEADER]: Becoming Leader. State: Replica: 460da7dff1754e7e8c5a18c885ef92df, State: Running, Role: LEADER
I20260619 14:59:30.325966 24369 consensus_queue.cc:237] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } }
I20260619 14:59:30.326790 24364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:59:30.329097 24370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "460da7dff1754e7e8c5a18c885ef92df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } } }
I20260619 14:59:30.329102 24371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 460da7dff1754e7e8c5a18c885ef92df. Latest consensus state: current_term: 1 leader_uuid: "460da7dff1754e7e8c5a18c885ef92df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } } }
I20260619 14:59:30.329567 24370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:30.329666 24371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:30.331259 24374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:59:30.336809 24374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:59:30.345180 24364 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 14:59:30.346459 24374 catalog_manager.cc:1347] Generated new cluster ID: 10721f2a0a124874b8cb693d734b95e7
I20260619 14:59:30.346591 24374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:59:30.394683 24374 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:59:30.396194 24374 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:59:30.420766 24374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: Generated new TSK 0
I20260619 14:59:30.422019 24374 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:59:31.182974 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:40623/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:44305
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 14:59:31 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781881171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 14:59:31.660419 24409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:31.660990 24409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:31.661273 24409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:31.674775 24409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:31.674965 24409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:31.675237 24409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:59:31.693032 24409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.175.212:40623/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:44305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:31.695039 24409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:31.697165 24409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:31.714982 24419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:31.716555 24420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:31.725143 24409 server_base.cc:1034] running on GCE node
W20260619 14:59:31.725554 24422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:31.726514 24409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:31.728174 24409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:31.729506 24409 hybrid_clock.cc:648] HybridClock initialized: now 1781881171729431 us; error 63 us; skew 500 ppm
I20260619 14:59:31.730027 24409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:31 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:31.734351 24409 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:59:31.740358 24409 webserver.cc:460] Webserver started at http://127.11.175.193:34065/ using document root <none> and password file <none>
I20260619 14:59:31.741345 24409 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:31.741494 24409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:31.741925 24409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 14:59:31 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:59:31.761960 24409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "a36f76697106467ba5ae1f88de87c85b"
format_stamp: "Formatted at 2026-06-19 14:59:31 on dist-test-slave-76rk"
server_key: "a71b9a49a1f4fce5509650be694c2d76"
server_key_iv: "77a2e69c0dff8b0ec4490c1523b74410"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:31.763988 24409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "a36f76697106467ba5ae1f88de87c85b"
format_stamp: "Formatted at 2026-06-19 14:59:31 on dist-test-slave-76rk"
server_key: "a71b9a49a1f4fce5509650be694c2d76"
server_key_iv: "77a2e69c0dff8b0ec4490c1523b74410"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:31.778620 24409 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.008s sys 0.004s
I20260619 14:59:31.811926 24429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:31.814142 24409 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.007s sys 0.007s
I20260619 14:59:31.814433 24409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "a36f76697106467ba5ae1f88de87c85b"
format_stamp: "Formatted at 2026-06-19 14:59:31 on dist-test-slave-76rk"
server_key: "a71b9a49a1f4fce5509650be694c2d76"
server_key_iv: "77a2e69c0dff8b0ec4490c1523b74410"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:31.814836 24409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:31.874770 24409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:31.880398 24409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:31.880826 24409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:31.883697 24409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:31.889161 24409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:31.889313 24409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:31.889489 24409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:31.889571 24409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:31.967859 24409 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:38741
I20260619 14:59:31.967902 24541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:38741 every 8 connection(s)
I20260619 14:59:31.969732 24409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:59:31.971534 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 24409
I20260619 14:59:31.971799 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:59:31.972817 11967 external_mini_cluster.cc:1428] Setting key 8d31b0638bded6cf7abc7a944366075c
Jun 19 14:59:32 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 14:59:32.019850 24545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:31.974155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:54571 (local address 127.11.175.254:44305)
0619 14:59:31.989336 (+ 15181us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:31.989347 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:31.993337 (+ 3990us) server_negotiation.cc:407] Connection header received
0619 14:59:31.993422 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:31.993428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:31.993558 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:31.993682 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:31.997257 (+ 3575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:31.998470 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:32.005343 (+ 6873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:32.006100 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:32.006230 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:32.011577 (+ 5347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:32.011598 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:32.011603 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:32.011650 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:32.015542 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:32.016754 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:32.016761 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:32.016765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:32.016939 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:32.017528 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:32.017533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:32.017536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:32.017922 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:32.018060 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:32.019161 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0619 14:59:32.019432 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14980,"thread_start_us":14854,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
I20260619 14:59:32.021868 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:31.973462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:44305 (local address 127.11.175.193:54571)
0619 14:59:31.990058 (+ 16596us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:31.990128 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:31.992855 (+ 2727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:31.995344 (+ 2489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:31.995362 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:31.996102 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:31.997049 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:31.997079 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:32.002804 (+ 5725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:32.002811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:32.005058 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:32.005096 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:32.006692 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:32.006703 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:32.007030 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:32.007942 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:32.007976 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:32.011235 (+ 3259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:32.015808 (+ 4573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:32.015819 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:32.015839 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:32.016576 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:32.017108 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:32.017113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:32.017117 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:32.017384 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:32.018157 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:32.018166 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:32.018688 (+ 522us) client_negotiation.cc:769] Sending connection context
0619 14:59:32.019595 (+ 907us) client_negotiation.cc:240] Negotiation successful
0619 14:59:32.020210 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16236,"thread_start_us":15993,"threads_started":1}
I20260619 14:59:32.026530 24542 heartbeater.cc:347] Connected to a master server at 127.11.175.254:44305
I20260619 14:59:32.026957 24542 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:32.028012 24542 heartbeater.cc:511] Master 127.11.175.254:44305 requested a full tablet report, sending...
I20260619 14:59:32.030799 24329 ts_manager.cc:194] Registered new tserver with Master: a36f76697106467ba5ae1f88de87c85b (127.11.175.193:38741)
I20260619 14:59:32.032985 24329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:54571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:59:32.071177 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:40623/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:44305
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 14:59:32 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
Jun 19 14:59:32 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
W20260619 14:59:32.562166 24548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:32.562726 24548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:32.562961 24548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:32.574890 24548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:32.575137 24548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:32.575368 24548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:59:32.591728 24548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.175.212:40623/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:44305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:32.593935 24548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:32.595903 24548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:32.632519 24555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:32.637553 24556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:32.641394 24559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:32.642012 24548 server_base.cc:1034] running on GCE node
I20260619 14:59:32.642899 24548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:32.654719 24548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:32.658275 24548 hybrid_clock.cc:648] HybridClock initialized: now 1781881172658142 us; error 118 us; skew 500 ppm
I20260619 14:59:32.658787 24548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:32 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:32.663188 24548 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:59:32.665271 24548 webserver.cc:460] Webserver started at http://127.11.175.194:43737/ using document root <none> and password file <none>
I20260619 14:59:32.666278 24548 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:32.666419 24548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:32.666808 24548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 14:59:32 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:59:32.685294 24548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "2ce8654d247546adbc74956f8ed0cc75"
format_stamp: "Formatted at 2026-06-19 14:59:32 on dist-test-slave-76rk"
server_key: "6bfe91c6dd58ceff4f36ea124e6cdcd1"
server_key_iv: "be5fcde2780d28a86b477a4523bc633d"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:32.686579 24548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "2ce8654d247546adbc74956f8ed0cc75"
format_stamp: "Formatted at 2026-06-19 14:59:32 on dist-test-slave-76rk"
server_key: "6bfe91c6dd58ceff4f36ea124e6cdcd1"
server_key_iv: "be5fcde2780d28a86b477a4523bc633d"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:32.693363 24548 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 14:59:32.716840 24572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:32.719184 24548 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.006s sys 0.002s
I20260619 14:59:32.719480 24548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "2ce8654d247546adbc74956f8ed0cc75"
format_stamp: "Formatted at 2026-06-19 14:59:32 on dist-test-slave-76rk"
server_key: "6bfe91c6dd58ceff4f36ea124e6cdcd1"
server_key_iv: "be5fcde2780d28a86b477a4523bc633d"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:32.719841 24548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:32.742177 24548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:32.748155 24548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:32.748574 24548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:32.749897 24548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:32.751878 24548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:32.752028 24548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 14:59:32.752200 24548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:32.752305 24548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:32.790069 24548 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:33775
I20260619 14:59:32.790122 24684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:33775 every 8 connection(s)
I20260619 14:59:32.792009 24548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:59:32.798959 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 24548
I20260619 14:59:32.799180 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:59:32.799875 11967 external_mini_cluster.cc:1428] Setting key 41d4bbecf772e4d5651cc0386446f6fb
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Jun 19 14:59:32 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:32.853391 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:32.795835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:45221 (local address 127.11.175.254:44305)
0619 14:59:32.812988 (+ 17153us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:32.812999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:32.821500 (+ 8501us) server_negotiation.cc:407] Connection header received
0619 14:59:32.823308 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:32.823314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:32.823424 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:32.823547 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:32.833344 (+ 9797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:32.834700 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:32.837750 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:32.838486 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:32.838610 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:32.844879 (+ 6269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:32.844890 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:32.844899 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:32.844954 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:32.848949 (+ 3995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:32.850182 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:32.850191 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:32.850197 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:32.850378 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:32.851024 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:32.851032 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:32.851037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:32.851465 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:32.851609 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:32.852620 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0619 14:59:32.852909 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16950,"thread_start_us":17145,"threads_started":1}
I20260619 14:59:32.854691 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:32.796448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:44305 (local address 127.11.175.194:45221)
0619 14:59:32.819565 (+ 23117us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:32.819660 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:32.822967 (+ 3307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:32.831185 (+ 8218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:32.831202 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:32.832063 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:32.833062 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:32.833094 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:32.834967 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:32.834974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:32.837511 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:32.837539 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:32.839034 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:32.839044 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:32.839405 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:32.840333 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:32.840373 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:32.844510 (+ 4137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:32.849150 (+ 4640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:32.849160 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:32.849183 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:32.849981 (+ 798us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:32.850547 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:32.850552 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:32.850556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:32.850883 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:32.851647 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:32.851656 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:32.852139 (+ 483us) client_negotiation.cc:769] Sending connection context
0619 14:59:32.852470 (+ 331us) client_negotiation.cc:240] Negotiation successful
0619 14:59:32.853222 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22662,"thread_start_us":22359,"threads_started":1}
I20260619 14:59:32.856485 24685 heartbeater.cc:347] Connected to a master server at 127.11.175.254:44305
I20260619 14:59:32.856845 24685 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:32.857780 24685 heartbeater.cc:511] Master 127.11.175.254:44305 requested a full tablet report, sending...
I20260619 14:59:32.859736 24329 ts_manager.cc:194] Registered new tserver with Master: 2ce8654d247546adbc74956f8ed0cc75 (127.11.175.194:33775)
I20260619 14:59:32.861027 24329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:45221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:59:32.879859 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:40623/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:44305
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:59:33.037274 24542 heartbeater.cc:503] Master 127.11.175.254:44305 was elected leader, sending a full tablet report...
W20260619 14:59:33.230168 24691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:33.230463 24691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:33.230558 24691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:33.238449 24691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:33.238571 24691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:33.238742 24691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:59:33.248973 24691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.175.212:40623/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:44305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:33.250244 24691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:33.251507 24691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:33.270282 24696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:33.270337 24697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:33.270363 24699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:33.272599 24691 server_base.cc:1034] running on GCE node
I20260619 14:59:33.273283 24691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:33.274253 24691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:33.275468 24691 hybrid_clock.cc:648] HybridClock initialized: now 1781881173275395 us; error 62 us; skew 500 ppm
I20260619 14:59:33.275832 24691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:33 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:33.279120 24691 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:59:33.280651 24691 webserver.cc:460] Webserver started at http://127.11.175.195:43787/ using document root <none> and password file <none>
I20260619 14:59:33.281334 24691 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:33.281437 24691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:33.281684 24691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 14:59:33 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:59:33.298067 24691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "cf67b74d72c8425390d2cfe9aafed130"
format_stamp: "Formatted at 2026-06-19 14:59:33 on dist-test-slave-76rk"
server_key: "320408a461f22fd08159e18fafbd3bbf"
server_key_iv: "a08515b9e612c6f1c80abb4ad6793264"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:33.298992 24691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "cf67b74d72c8425390d2cfe9aafed130"
format_stamp: "Formatted at 2026-06-19 14:59:33 on dist-test-slave-76rk"
server_key: "320408a461f22fd08159e18fafbd3bbf"
server_key_iv: "a08515b9e612c6f1c80abb4ad6793264"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:33.304615 24691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 14:59:33.322666 24706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:33.324342 24691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:59:33.324563 24691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "cf67b74d72c8425390d2cfe9aafed130"
format_stamp: "Formatted at 2026-06-19 14:59:33 on dist-test-slave-76rk"
server_key: "320408a461f22fd08159e18fafbd3bbf"
server_key_iv: "a08515b9e612c6f1c80abb4ad6793264"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:33.324816 24691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:33.342556 24691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:33.346850 24691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:33.347223 24691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:33.348248 24691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:33.349819 24691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:33.349921 24691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:33.350075 24691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:33.350147 24691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:33.383463 24691 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:45597
I20260619 14:59:33.383509 24818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:45597 every 8 connection(s)
I20260619 14:59:33.384862 24691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:59:33.388314 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 24691
I20260619 14:59:33.388507 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:59:33.389001 11967 external_mini_cluster.cc:1428] Setting key 182e228e4bd805faab73cba585971195
Jun 19 14:59:33 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:33.405779 24821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:33.387994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:38227 (local address 127.11.175.254:44305)
0619 14:59:33.388568 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:33.388577 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:33.389964 (+ 1387us) server_negotiation.cc:407] Connection header received
0619 14:59:33.391590 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:33.391595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:33.391687 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:33.391810 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:33.393212 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:33.393974 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:33.395616 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:33.396043 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:33.396175 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:33.399630 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:33.399639 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:33.399648 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:33.399682 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:33.402333 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:33.403263 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:33.403269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:33.403273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:33.403417 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:33.403909 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:33.403914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:33.403917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:33.404227 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:33.404430 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:33.405159 (+ 729us) server_negotiation.cc:299] Negotiation successful
0619 14:59:33.405414 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":271,"threads_started":1}
I20260619 14:59:33.406857 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:33.388638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:44305 (local address 127.11.175.195:38227)
0619 14:59:33.389499 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:33.389544 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:33.391297 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:33.392051 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:33.392060 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:33.392486 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:33.393051 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:33.393067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:33.394135 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:33.394139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:33.395464 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:33.395476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:33.396283 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:33.396303 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:33.396555 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:33.397245 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:33.397272 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:33.399388 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:33.402484 (+ 3096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:33.402494 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:33.402512 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:33.403127 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:33.403542 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:33.403547 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:33.403551 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:33.403786 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:33.404362 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:33.404394 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:33.404769 (+ 375us) client_negotiation.cc:769] Sending connection context
0619 14:59:33.405058 (+ 289us) client_negotiation.cc:240] Negotiation successful
0619 14:59:33.405627 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":382,"threads_started":1}
I20260619 14:59:33.408623 24819 heartbeater.cc:347] Connected to a master server at 127.11.175.254:44305
I20260619 14:59:33.408977 24819 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:33.409713 24819 heartbeater.cc:511] Master 127.11.175.254:44305 requested a full tablet report, sending...
I20260619 14:59:33.411311 24329 ts_manager.cc:194] Registered new tserver with Master: cf67b74d72c8425390d2cfe9aafed130 (127.11.175.195:45597)
I20260619 14:59:33.412154 24329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:38227
I20260619 14:59:33.416975 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 14:59:33.418766 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 24409
Jun 19 14:59:33 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:59:33.437552 24824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:33.419853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.11.175.193:38741)
0619 14:59:33.420540 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:33.420547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:33.420571 (+ 24us) server_negotiation.cc:407] Connection header received
0619 14:59:33.420632 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:33.420638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:33.420931 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:33.421110 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:33.422801 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:33.423816 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:33.425595 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:33.426167 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:33.426300 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:33.431175 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:33.431189 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:33.431251 (+ 62us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:33.431311 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:33.434378 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:33.435358 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:33.435365 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:33.435367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:33.435524 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:33.435944 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:33.435968 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:33.435974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:33.436358 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:33.436584 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:33.436891 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 14:59:33.437149 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":353,"threads_started":1}
I20260619 14:59:33.438966 24537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:33.521520 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 24409
I20260619 14:59:33.538250 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 24548
Jun 19 14:59:33 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:59:33.555428 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:33.539225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51040 (local address 127.11.175.194:33775)
0619 14:59:33.539766 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:33.539773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:33.539799 (+ 26us) server_negotiation.cc:407] Connection header received
0619 14:59:33.539859 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:33.539865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:33.540076 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:33.540243 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:33.541592 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:33.542573 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:33.544050 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:33.544615 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:33.544757 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:33.549483 (+ 4726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:33.549496 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:33.549513 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:33.549560 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:33.552314 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:33.553082 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:33.553090 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:33.553092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:33.553280 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:33.553688 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:33.553694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:33.553698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:33.554057 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:33.554406 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:33.554727 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 14:59:33.555038 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":283,"threads_started":1}
I20260619 14:59:33.556922 24680 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:33.639685 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 24548
I20260619 14:59:33.655247 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 24691
Jun 19 14:59:33 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:59:33.672326 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:33.656176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.11.175.195:45597)
0619 14:59:33.656832 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:33.656838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:33.656866 (+ 28us) server_negotiation.cc:407] Connection header received
0619 14:59:33.656924 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:33.656931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:33.657149 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:33.657334 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:33.658749 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:33.659819 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:33.661186 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:33.661762 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:33.661916 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:33.666446 (+ 4530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:33.666458 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:33.666474 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:33.666511 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:33.669285 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:33.670020 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:33.670028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:33.670030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:33.670164 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:33.670567 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:33.670573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:33.670576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:33.671012 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:33.671352 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:33.671680 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 14:59:33.671969 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":341,"threads_started":1}
I20260619 14:59:33.673715 24814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:33.757638 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 24691
I20260619 14:59:33.773288 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 24297
I20260619 14:59:33.774895 24359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:33.884420 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 24297
I20260619 14:59:33.906445 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:44305
--webserver_interface=127.11.175.254
--webserver_port=41269
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:44305
--ranger_config_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:40623/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.175.212:40623/kms with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:59:34.244848 24831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:34.245234 24831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:34.245410 24831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:34.252949 24831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:59:34.253103 24831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:34.253218 24831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:34.253319 24831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:59:34.253403 24831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:59:34.264074 24831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.175.212:40623/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:44305
--ranger_config_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:44305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=41269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:34.265447 24831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:34.266876 24831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:34.279466 24836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:34.279449 24837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:34.279466 24839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:34.280274 24831 server_base.cc:1034] running on GCE node
I20260619 14:59:34.281059 24831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:34.282454 24831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:34.283668 24831 hybrid_clock.cc:648] HybridClock initialized: now 1781881174283597 us; error 63 us; skew 500 ppm
I20260619 14:59:34.284052 24831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:34 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881174, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:34.287464 24831 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:59:34.289070 24831 webserver.cc:460] Webserver started at http://127.11.175.254:41269/ using document root <none> and password file <none>
I20260619 14:59:34.289747 24831 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:34.289891 24831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 14:59:34 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881174, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:59:34.340065 24831 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260619 14:59:34.343248 24847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:34.344761 24831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 14:59:34.344951 24831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "460da7dff1754e7e8c5a18c885ef92df"
format_stamp: "Formatted at 2026-06-19 14:59:30 on dist-test-slave-76rk"
server_key: "d8077ca3b09095ee4e2873ff05aa63bd"
server_key_iv: "61782ebd72628d265657baaafa872d2d"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:34.345929 24831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:34.368665 24831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:34.372653 24831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:34.372947 24831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:34.392117 24831 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:44305
I20260619 14:59:34.392160 24898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:44305 every 8 connection(s)
I20260619 14:59:34.393710 24831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:59:34.400942 24899 sys_catalog.cc:263] Verifying existing consensus state
I20260619 14:59:34.402176 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 24831
I20260619 14:59:34.403477 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:40623/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:38741
--local_ip_for_outbound_sockets=127.11.175.193
--tserver_master_addrs=127.11.175.254:44305
--webserver_port=34065
--webserver_interface=127.11.175.193
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.175.212:40623/kms with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:59:34.406040 24899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: Bootstrap starting.
I20260619 14:59:34.427137 24899 log.cc:824] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:34.436162 24899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 14:59:34.436723 24899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: Bootstrap complete.
I20260619 14:59:34.445765 24899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } }
I20260619 14:59:34.446708 24899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460da7dff1754e7e8c5a18c885ef92df, State: Initialized, Role: FOLLOWER
I20260619 14:59:34.447366 24899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } }
I20260619 14:59:34.447594 24899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:59:34.447732 24899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:59:34.447916 24899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 1 FOLLOWER]: Advancing to term 2
I20260619 14:59:34.452273 24899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } }
I20260619 14:59:34.452790 24899 leader_election.cc:302] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 460da7dff1754e7e8c5a18c885ef92df; no voters:
I20260619 14:59:34.453439 24899 leader_election.cc:288] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 14:59:34.453583 24903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 2 FOLLOWER]: Leader election won for term 2
I20260619 14:59:34.454782 24903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [term 2 LEADER]: Becoming Leader. State: Replica: 460da7dff1754e7e8c5a18c885ef92df, State: Running, Role: LEADER
I20260619 14:59:34.455425 24903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } }
I20260619 14:59:34.455753 24899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:59:34.458194 24905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 460da7dff1754e7e8c5a18c885ef92df. Latest consensus state: current_term: 2 leader_uuid: "460da7dff1754e7e8c5a18c885ef92df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } } }
I20260619 14:59:34.458678 24905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:34.459224 24904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "460da7dff1754e7e8c5a18c885ef92df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460da7dff1754e7e8c5a18c885ef92df" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 44305 } } }
I20260619 14:59:34.459527 24904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:34.472069 24908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:59:34.478101 24908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:59:34.479627 24908 catalog_manager.cc:1259] Loaded cluster ID: 10721f2a0a124874b8cb693d734b95e7
I20260619 14:59:34.479867 24908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:59:34.483950 24908 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:59:34.485616 24908 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: Loaded TSK: 0
I20260619 14:59:34.487283 24908 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 14:59:34.492103 24899 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260619 14:59:34.852428 24901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:34.852771 24901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:34.852936 24901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:34.864504 24901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:34.864728 24901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:34.864984 24901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:59:34.880772 24901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.175.212:40623/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:38741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=34065
--tserver_master_addrs=127.11.175.254:44305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:34.882499 24901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:34.884214 24901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:34.903115 24937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:34.903501 24936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:34.909842 24901 server_base.cc:1034] running on GCE node
W20260619 14:59:34.910797 24939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:34.911818 24901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:34.935937 24901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:34.937397 24901 hybrid_clock.cc:648] HybridClock initialized: now 1781881174937320 us; error 85 us; skew 500 ppm
I20260619 14:59:34.937920 24901 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:34 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881174, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:34.942752 24901 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:59:34.959107 24901 webserver.cc:460] Webserver started at http://127.11.175.193:34065/ using document root <none> and password file <none>
I20260619 14:59:34.960280 24901 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:34.960559 24901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 14:59:34 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881174, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:59:35.023689 24901 fs_manager.cc:709] Time spent opening directory manager: real 0.043s user 0.026s sys 0.017s
I20260619 14:59:35.033716 24947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:35.035972 24901 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.004s
I20260619 14:59:35.036276 24901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "a36f76697106467ba5ae1f88de87c85b"
format_stamp: "Formatted at 2026-06-19 14:59:31 on dist-test-slave-76rk"
server_key: "a71b9a49a1f4fce5509650be694c2d76"
server_key_iv: "77a2e69c0dff8b0ec4490c1523b74410"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:35.037947 24901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:35.092168 24901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:35.098153 24901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:35.098532 24901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:35.099956 24901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:35.101847 24901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:35.101966 24901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:35.102082 24901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:35.102170 24901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:35.160437 24901 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:38741
I20260619 14:59:35.160522 25060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:38741 every 8 connection(s)
I20260619 14:59:35.162530 24901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:59:35.164326 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 24901
I20260619 14:59:35.165210 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:40623/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:33775
--local_ip_for_outbound_sockets=127.11.175.194
--tserver_master_addrs=127.11.175.254:44305
--webserver_port=43737
--webserver_interface=127.11.175.194
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.175.212:40623/kms with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 14:59:35 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881174, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:35.202011 25064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:35.170519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:44305 (local address 127.11.175.193:48181)
0619 14:59:35.176952 (+ 6433us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:35.177008 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:35.179843 (+ 2835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:35.182668 (+ 2825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:35.182681 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:35.183264 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:35.183868 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:35.183886 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:35.186767 (+ 2881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:35.186774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:35.188059 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:35.188071 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:35.189439 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:35.189443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:35.189632 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:35.190340 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:35.190364 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:35.192497 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:35.196788 (+ 4291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:35.196798 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:35.196816 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:35.197299 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:35.197784 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:35.197789 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:35.197793 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:35.198006 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:35.199240 (+ 1234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:35.199249 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:35.199671 (+ 422us) client_negotiation.cc:769] Sending connection context
0619 14:59:35.199973 (+ 302us) client_negotiation.cc:240] Negotiation successful
0619 14:59:35.200557 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6052,"thread_start_us":9991,"threads_started":1}
I20260619 14:59:35.205070 25063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:35.170282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:48181 (local address 127.11.175.254:44305)
0619 14:59:35.180877 (+ 10595us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:35.180900 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:35.180952 (+ 52us) server_negotiation.cc:407] Connection header received
0619 14:59:35.181490 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:35.181512 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:35.182032 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:35.182505 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:35.185214 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:35.186570 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:35.188223 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:35.188956 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:35.189129 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:35.192780 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:35.192789 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:35.192805 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:35.192848 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:35.196581 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:35.197471 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:35.197479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:35.197485 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:35.197650 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:35.198134 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:35.198144 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:35.198146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:35.198859 (+ 713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:35.202743 (+ 3884us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:35.203086 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 14:59:35.203579 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10293,"thread_start_us":10105,"threads_started":1}
I20260619 14:59:35.209003 25061 heartbeater.cc:347] Connected to a master server at 127.11.175.254:44305
I20260619 14:59:35.209375 25061 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:35.210350 25061 heartbeater.cc:511] Master 127.11.175.254:44305 requested a full tablet report, sending...
W20260619 14:59:35.214033 25061 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:44305 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 14:59:35.214263 25061 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:35.214555 25061 heartbeater.cc:511] Master 127.11.175.254:44305 requested a full tablet report, sending...
I20260619 14:59:35.215906 25061 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:35.216194 25061 heartbeater.cc:511] Master 127.11.175.254:44305 requested a full tablet report, sending...
W20260619 14:59:35.217314 25061 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 14:59:35.561932 25065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:35.562207 25065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:35.562361 25065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:35.569905 25065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:35.570014 25065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:35.570180 25065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:59:35.580760 25065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.175.212:40623/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:33775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=43737
--tserver_master_addrs=127.11.175.254:44305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:35.582213 25065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:35.583604 25065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:35.597774 25078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:35.597793 25077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:35.598591 25080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:35.599668 25065 server_base.cc:1034] running on GCE node
I20260619 14:59:35.600478 25065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:35.601758 25065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:35.603014 25065 hybrid_clock.cc:648] HybridClock initialized: now 1781881175602944 us; error 45 us; skew 500 ppm
I20260619 14:59:35.603454 25065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:35 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:35.607596 25065 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:59:35.609681 25065 webserver.cc:460] Webserver started at http://127.11.175.194:43737/ using document root <none> and password file <none>
I20260619 14:59:35.610441 25065 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:35.610610 25065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 14:59:35 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:59:35.647544 25065 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.004s sys 0.004s
I20260619 14:59:35.650926 25090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:35.652563 25065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:59:35.652779 25065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "2ce8654d247546adbc74956f8ed0cc75"
format_stamp: "Formatted at 2026-06-19 14:59:32 on dist-test-slave-76rk"
server_key: "6bfe91c6dd58ceff4f36ea124e6cdcd1"
server_key_iv: "be5fcde2780d28a86b477a4523bc633d"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:35.653865 25065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:35.679807 25065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:35.684463 25065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:35.684769 25065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:35.696472 25065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:35.704876 25065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:35.705098 25065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 14:59:35.705278 25065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:35.705435 25065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:35.781505 25065 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:33775
I20260619 14:59:35.781584 25204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:33775 every 8 connection(s)
I20260619 14:59:35.783417 25065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:59:35.793442 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 25065
I20260619 14:59:35.794411 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.175.212:40623/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:45597
--local_ip_for_outbound_sockets=127.11.175.195
--tserver_master_addrs=127.11.175.254:44305
--webserver_port=43787
--webserver_interface=127.11.175.195
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.175.212:40623/kms with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 14:59:35 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
Jun 19 14:59:35 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781881175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:35.820459 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:35.787652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:43903 (local address 127.11.175.254:44305)
0619 14:59:35.799182 (+ 11530us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:35.799189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:35.799215 (+ 26us) server_negotiation.cc:407] Connection header received
0619 14:59:35.799296 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:35.799302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:35.799395 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:35.799535 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:35.802097 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:35.803312 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:35.806200 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:35.806942 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:35.807093 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:35.812460 (+ 5367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:35.812469 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:35.812474 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:35.812538 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:35.816247 (+ 3709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:35.817373 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:35.817379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:35.817383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:35.817544 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:35.818170 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:35.818177 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:35.818183 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:35.818575 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:35.818733 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:35.819841 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0619 14:59:35.820081 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11358,"thread_start_us":11242,"threads_started":1}
I20260619 14:59:35.823031 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:35.787005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:44305 (local address 127.11.175.194:43903)
0619 14:59:35.793254 (+ 6249us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:35.793296 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:35.795203 (+ 1907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:35.800119 (+ 4916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:35.800135 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:35.800882 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:35.801812 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:35.801859 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:35.803577 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:35.803582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:35.805978 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:35.806006 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:35.807449 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:35.807458 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:35.807804 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:35.808793 (+ 989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:35.808828 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:35.812111 (+ 3283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:35.816488 (+ 4377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:35.816499 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:35.816521 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:35.817206 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:35.817710 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:35.817718 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:35.817722 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:35.818030 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:35.818864 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:35.818873 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:35.819364 (+ 491us) client_negotiation.cc:769] Sending connection context
0619 14:59:35.820797 (+ 1433us) client_negotiation.cc:240] Negotiation successful
0619 14:59:35.821444 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5948,"thread_start_us":5685,"threads_started":1}
I20260619 14:59:35.825016 25205 heartbeater.cc:347] Connected to a master server at 127.11.175.254:44305
I20260619 14:59:35.825376 25205 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:35.826344 25205 heartbeater.cc:511] Master 127.11.175.254:44305 requested a full tablet report, sending...
I20260619 14:59:35.828545 24864 ts_manager.cc:194] Registered new tserver with Master: 2ce8654d247546adbc74956f8ed0cc75 (127.11.175.194:33775)
I20260619 14:59:35.830979 24864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:43903
I20260619 14:59:36.217692 25061 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:36.218238 25061 heartbeater.cc:511] Master 127.11.175.254:44305 requested a full tablet report, sending...
I20260619 14:59:36.219756 24864 ts_manager.cc:194] Registered new tserver with Master: a36f76697106467ba5ae1f88de87c85b (127.11.175.193:38741)
I20260619 14:59:36.221109 24864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:48181
Jun 19 14:59:36 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
W20260619 14:59:36.289839 25209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:36.290251 25209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:36.290391 25209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:36.302103 25209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:36.302263 25209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:36.302433 25209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:59:36.317934 25209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.175.212:40623/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:45597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=43787
--tserver_master_addrs=127.11.175.254:44305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:36.319649 25209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:36.321311 25209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:36.338505 25215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:36.339332 25216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:36.345438 25218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:36.345731 25209 server_base.cc:1034] running on GCE node
I20260619 14:59:36.346817 25209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:36.348083 25209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:36.349324 25209 hybrid_clock.cc:648] HybridClock initialized: now 1781881176349255 us; error 53 us; skew 500 ppm
I20260619 14:59:36.349704 25209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:36 dist-test-slave-76rk krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881176, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:36.353999 25209 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:59:36.362553 25209 webserver.cc:460] Webserver started at http://127.11.175.195:43787/ using document root <none> and password file <none>
I20260619 14:59:36.363590 25209 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:36.363757 25209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 14:59:36 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881176, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
I20260619 14:59:36.401207 25209 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.003s sys 0.007s
I20260619 14:59:36.405088 25226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:36.407176 25209 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260619 14:59:36.407466 25209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "cf67b74d72c8425390d2cfe9aafed130"
format_stamp: "Formatted at 2026-06-19 14:59:33 on dist-test-slave-76rk"
server_key: "320408a461f22fd08159e18fafbd3bbf"
server_key_iv: "a08515b9e612c6f1c80abb4ad6793264"
server_key_version: "kuduclusterkey@0"
I20260619 14:59:36.409181 25209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:36.451893 25209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:36.457867 25209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:36.458285 25209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:36.459766 25209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:36.461839 25209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:36.462028 25209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 14:59:36.462227 25209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:36.462368 25209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:36.548573 25209 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:45597
I20260619 14:59:36.550873 25209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:59:36.551553 25339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:45597 every 8 connection(s)
I20260619 14:59:36.558552 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 25209
Jun 19 14:59:36 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for HTTP/127.11.175.212@KRBTEST.COM
Jun 19 14:59:36 dist-test-slave-76rk krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881176, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:36.610543 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:36.559820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.11.175.254:44305)
0619 14:59:36.575711 (+ 15891us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:36.575721 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:36.587267 (+ 11546us) server_negotiation.cc:407] Connection header received
0619 14:59:36.587679 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:36.587689 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:36.587846 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:36.587955 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:36.590120 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.591414 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.597668 (+ 6254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.598305 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.598408 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:36.602462 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:36.602472 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:36.602481 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:36.602531 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:36.606421 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:36.608172 (+ 1751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:36.608177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:36.608180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:36.608334 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:36.608780 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:36.608785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:36.608787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:36.609107 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:36.609357 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:36.609946 (+ 589us) server_negotiation.cc:299] Negotiation successful
0619 14:59:36.610169 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15681,"thread_start_us":36982,"threads_started":1}
I20260619 14:59:36.623867 25342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:36.567253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:49193 (local address 127.11.175.254:44305)
0619 14:59:36.597101 (+ 29848us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:36.597108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:36.597125 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:59:36.597196 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:36.597202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:36.597306 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:36.597428 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:36.599956 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.601139 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.603738 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.604453 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.605447 (+ 994us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:36.616048 (+ 10601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:36.616055 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:36.616060 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:36.616102 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:36.619474 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:36.620599 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:36.620604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:36.620607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:36.620751 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:36.621342 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:36.621347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:36.621350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:36.621772 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:36.622961 (+ 1189us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:36.623128 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 14:59:36.623438 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29701,"thread_start_us":15402,"threads_started":1}
I20260619 14:59:36.625896 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:36.566276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:44305 (local address 127.11.175.195:49193)
0619 14:59:36.571092 (+ 4816us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:36.571163 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:36.574620 (+ 3457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:36.597885 (+ 23265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:36.597901 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:36.598673 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:36.599672 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:36.599704 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.601353 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.601359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:36.603488 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:36.603514 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.604718 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.604726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:36.605027 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:36.605731 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:36.605765 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:36.613390 (+ 7625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:36.619676 (+ 6286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:36.619687 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:36.619707 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:36.620421 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:36.620884 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:36.620889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:36.620892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:36.621177 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:36.621952 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:36.621961 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:36.622485 (+ 524us) client_negotiation.cc:769] Sending connection context
0619 14:59:36.622910 (+ 425us) client_negotiation.cc:240] Negotiation successful
0619 14:59:36.623544 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4430,"thread_start_us":12233,"threads_started":1}
I20260619 14:59:36.628403 25340 heartbeater.cc:347] Connected to a master server at 127.11.175.254:44305
I20260619 14:59:36.628736 25340 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:36.629621 25340 heartbeater.cc:511] Master 127.11.175.254:44305 requested a full tablet report, sending...
I20260619 14:59:36.632438 24863 ts_manager.cc:194] Registered new tserver with Master: cf67b74d72c8425390d2cfe9aafed130 (127.11.175.195:45597)
I20260619 14:59:36.633754 24863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:49193
I20260619 14:59:36.641491 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 14:59:36.671896 25342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:36.654022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60760 (local address 127.11.175.254:44305)
0619 14:59:36.654235 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:36.654244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:36.654880 (+ 636us) server_negotiation.cc:407] Connection header received
0619 14:59:36.655173 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:36.655179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:36.655285 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:36.655364 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:36.657589 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.658759 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.660726 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.661370 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.661480 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:36.665052 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:36.665058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:36.665062 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:36.665105 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:36.668525 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:36.669427 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:36.669432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:36.669436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:36.669583 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:36.670068 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:36.670073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:36.670077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:36.670453 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:36.670688 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:36.671289 (+ 601us) server_negotiation.cc:299] Negotiation successful
0619 14:59:36.671544 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 14:59:36.678961 24863 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:59:36.690388 24863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:59:36.745020 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:36.729782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38741 (local address 127.0.0.1:34396)
0619 14:59:36.734139 (+ 4357us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:36.734194 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:36.734423 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:36.735668 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:36.735679 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:36.735720 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:36.736222 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:36.736249 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.738504 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.738510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:36.741086 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:36.741103 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.743919 (+ 2816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.743925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:36.744149 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:36.744213 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 14:59:36.744364 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 14:59:36.744584 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4081,"thread_start_us":8331,"threads_started":1}
I20260619 14:59:36.746253 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:36.730350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34396 (local address 127.11.175.193:38741)
0619 14:59:36.733533 (+ 3183us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:36.733543 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:36.734696 (+ 1153us) server_negotiation.cc:407] Connection header received
0619 14:59:36.734813 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:36.734821 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:36.735108 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:36.735331 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:36.736536 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.738324 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.741360 (+ 3036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.743310 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.743531 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:36.743735 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:36.745607 (+ 1872us) server_negotiation.cc:299] Negotiation successful
0619 14:59:36.745831 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2979,"thread_start_us":6311,"threads_started":1}
I20260619 14:59:36.752599 24996 tablet_service.cc:1467] Processing CreateTablet for tablet fe50b0972f9540fa90e084377dd951df (DEFAULT_TABLE table=test-table [id=fc4ff75d8285461fb1b27afc99ff62ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:59:36.754307 24996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe50b0972f9540fa90e084377dd951df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:36.756083 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:36.733509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:45597 (local address 127.0.0.1:57414)
0619 14:59:36.745296 (+ 11787us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:36.745315 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:36.745419 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:36.747920 (+ 2501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:36.747926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:36.747952 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:36.748134 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:36.748146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.751539 (+ 3393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.751545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:36.753352 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:36.753369 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.755290 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.755297 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:36.755496 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:36.755533 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 14:59:36.755605 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 14:59:36.755766 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11697,"thread_start_us":18508,"threads_started":1}
I20260619 14:59:36.757198 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:36.734082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57414 (local address 127.11.175.195:45597)
0619 14:59:36.746958 (+ 12876us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:36.746966 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:36.747000 (+ 34us) server_negotiation.cc:407] Connection header received
0619 14:59:36.747094 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:36.747102 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:36.747393 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:36.747608 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:36.749825 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.751386 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.753578 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.755090 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.756420 (+ 1330us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:36.756615 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:36.756686 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 14:59:36.756844 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12698,"thread_start_us":18120,"threads_started":1}
I20260619 14:59:36.761487 25274 tablet_service.cc:1467] Processing CreateTablet for tablet fe50b0972f9540fa90e084377dd951df (DEFAULT_TABLE table=test-table [id=fc4ff75d8285461fb1b27afc99ff62ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:59:36.764874 25274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe50b0972f9540fa90e084377dd951df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:36.765300 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:36.732864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:33775 (local address 127.0.0.1:51054)
0619 14:59:36.743997 (+ 11133us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:36.744019 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:36.744147 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:36.753475 (+ 9328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:36.753482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:36.753513 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:36.753853 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:36.753864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.758647 (+ 4783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.758654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:36.760679 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:36.760698 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.764370 (+ 3672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.764378 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:36.764566 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:36.764601 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 14:59:36.764682 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 14:59:36.764836 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10976,"thread_start_us":13789,"threads_started":1}
I20260619 14:59:36.767252 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:36.733069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51054 (local address 127.11.175.194:33775)
0619 14:59:36.748934 (+ 15865us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:36.748942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:36.748972 (+ 30us) server_negotiation.cc:407] Connection header received
0619 14:59:36.749064 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:36.749071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:36.749311 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:36.749522 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:36.756058 (+ 6536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.758425 (+ 2367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.762383 (+ 3958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:36.763883 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:36.764047 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:36.764213 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:36.766669 (+ 2456us) server_negotiation.cc:299] Negotiation successful
0619 14:59:36.766929 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15697,"thread_start_us":24336,"threads_started":1}
I20260619 14:59:36.777730 25140 tablet_service.cc:1467] Processing CreateTablet for tablet fe50b0972f9540fa90e084377dd951df (DEFAULT_TABLE table=test-table [id=fc4ff75d8285461fb1b27afc99ff62ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:59:36.780093 25361 tablet_bootstrap.cc:492] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: Bootstrap starting.
I20260619 14:59:36.784030 25361 tablet_bootstrap.cc:654] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:36.785375 25361 log.cc:824] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:36.790824 25361 tablet_bootstrap.cc:492] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: No bootstrap required, opened a new log
I20260619 14:59:36.791064 25140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe50b0972f9540fa90e084377dd951df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:36.791153 25361 ts_tablet_manager.cc:1397] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260619 14:59:36.801743 25361 raft_consensus.cc:348] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 } } peers { permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 } } peers { permanent_uuid: "cf67b74d72c8425390d2cfe9aafed130" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 45597 } }
I20260619 14:59:36.802245 25361 raft_consensus.cc:374] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:36.802386 25361 raft_consensus.cc:729] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a36f76697106467ba5ae1f88de87c85b, State: Initialized, Role: FOLLOWER
I20260619 14:59:36.803186 25361 consensus_queue.cc:260] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 } } peers { permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 } } peers { permanent_uuid: "cf67b74d72c8425390d2cfe9aafed130" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 45597 } }
I20260619 14:59:36.811553 25361 ts_tablet_manager.cc:1428] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: Time spent starting tablet: real 0.020s user 0.019s sys 0.000s
I20260619 14:59:36.811966 25061 heartbeater.cc:503] Master 127.11.175.254:44305 was elected leader, sending a full tablet report...
I20260619 14:59:36.816042 25364 tablet_bootstrap.cc:492] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: Bootstrap starting.
I20260619 14:59:36.820097 25364 tablet_bootstrap.cc:654] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:36.821552 25364 log.cc:824] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:36.826813 25365 tablet_bootstrap.cc:492] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: Bootstrap starting.
I20260619 14:59:36.829020 25364 tablet_bootstrap.cc:492] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: No bootstrap required, opened a new log
I20260619 14:59:36.829311 25364 ts_tablet_manager.cc:1397] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260619 14:59:36.837682 25205 heartbeater.cc:503] Master 127.11.175.254:44305 was elected leader, sending a full tablet report...
I20260619 14:59:36.838579 25365 tablet_bootstrap.cc:654] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:36.839998 25365 log.cc:824] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:36.849491 25365 tablet_bootstrap.cc:492] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: No bootstrap required, opened a new log
I20260619 14:59:36.849818 25365 ts_tablet_manager.cc:1397] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.000s
I20260619 14:59:36.852222 25364 raft_consensus.cc:348] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 } } peers { permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 } } peers { permanent_uuid: "cf67b74d72c8425390d2cfe9aafed130" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 45597 } }
I20260619 14:59:36.852725 25364 raft_consensus.cc:374] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:36.852840 25364 raft_consensus.cc:729] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf67b74d72c8425390d2cfe9aafed130, State: Initialized, Role: FOLLOWER
I20260619 14:59:36.853618 25364 consensus_queue.cc:260] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 } } peers { permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 } } peers { permanent_uuid: "cf67b74d72c8425390d2cfe9aafed130" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 45597 } }
I20260619 14:59:36.861135 25365 raft_consensus.cc:348] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 } } peers { permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 } } peers { permanent_uuid: "cf67b74d72c8425390d2cfe9aafed130" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 45597 } }
I20260619 14:59:36.861652 25365 raft_consensus.cc:374] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:36.861778 25365 raft_consensus.cc:729] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce8654d247546adbc74956f8ed0cc75, State: Initialized, Role: FOLLOWER
I20260619 14:59:36.862578 25365 consensus_queue.cc:260] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 } } peers { permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 } } peers { permanent_uuid: "cf67b74d72c8425390d2cfe9aafed130" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 45597 } }
I20260619 14:59:36.864482 25340 heartbeater.cc:503] Master 127.11.175.254:44305 was elected leader, sending a full tablet report...
I20260619 14:59:36.865145 25364 ts_tablet_manager.cc:1428] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: Time spent starting tablet: real 0.036s user 0.008s sys 0.017s
I20260619 14:59:36.873204 25365 ts_tablet_manager.cc:1428] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: Time spent starting tablet: real 0.023s user 0.015s sys 0.004s
W20260619 14:59:36.918840 25062 tablet.cc:2369] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 14:59:37.041957 25206 tablet.cc:2369] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 14:59:37.089128 25345 tablet.cc:2369] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:59:37.140890 25370 raft_consensus.cc:482] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:59:37.141161 25370 raft_consensus.cc:504] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 } } peers { permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 } } peers { permanent_uuid: "cf67b74d72c8425390d2cfe9aafed130" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 45597 } }
I20260619 14:59:37.143011 25370 leader_election.cc:288] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a36f76697106467ba5ae1f88de87c85b (127.11.175.193:38741), 2ce8654d247546adbc74956f8ed0cc75 (127.11.175.194:33775)
I20260619 14:59:37.151687 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:37.143384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:35635 (local address 127.11.175.194:33775)
0619 14:59:37.143627 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:37.143634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:37.144212 (+ 578us) server_negotiation.cc:407] Connection header received
0619 14:59:37.144478 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:37.144484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:37.144567 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:37.144666 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:37.145519 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.146659 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.149415 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.150688 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.150863 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:37.150946 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:37.151288 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 14:59:37.151420 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260619 14:59:37.151868 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:37.143270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:33775 (local address 127.11.175.195:35635)
0619 14:59:37.144083 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:37.144105 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:37.144259 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:37.144728 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:37.144734 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:37.144853 (+ 119us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:37.145316 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:37.145329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.146802 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.146807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:37.149221 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:37.149239 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.150866 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.150871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:37.151088 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:37.151120 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 14:59:37.151242 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 14:59:37.151389 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":91,"spinlock_wait_cycles":11008,"thread_start_us":540,"threads_started":1}
I20260619 14:59:37.151871 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:37.143516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:58925 (local address 127.11.175.193:38741)
0619 14:59:37.143700 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:37.143707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:37.144291 (+ 584us) server_negotiation.cc:407] Connection header received
0619 14:59:37.144468 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:37.144474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:37.144563 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:37.144666 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:37.145518 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.146659 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.149415 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.150761 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.150912 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:37.150984 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:37.151466 (+ 482us) server_negotiation.cc:299] Negotiation successful
0619 14:59:37.151599 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 14:59:37.152377 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:37.143250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38741 (local address 127.11.175.195:58925)
0619 14:59:37.144083 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:37.144105 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:37.144321 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:37.144739 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:37.144743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:37.144853 (+ 110us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:37.145364 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:37.145373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.146802 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.146807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:37.149221 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:37.149239 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.151115 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.151119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:37.151283 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:37.151327 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 14:59:37.151436 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 14:59:37.151570 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":223,"spinlock_wait_cycles":181504,"thread_start_us":438,"threads_started":1}
I20260619 14:59:37.153074 25160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe50b0972f9540fa90e084377dd951df" candidate_uuid: "cf67b74d72c8425390d2cfe9aafed130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce8654d247546adbc74956f8ed0cc75" is_pre_election: true
I20260619 14:59:37.153390 25016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe50b0972f9540fa90e084377dd951df" candidate_uuid: "cf67b74d72c8425390d2cfe9aafed130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a36f76697106467ba5ae1f88de87c85b" is_pre_election: true
I20260619 14:59:37.153529 25160 raft_consensus.cc:2454] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf67b74d72c8425390d2cfe9aafed130 in term 0.
I20260619 14:59:37.153854 25016 raft_consensus.cc:2454] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf67b74d72c8425390d2cfe9aafed130 in term 0.
I20260619 14:59:37.154175 25229 leader_election.cc:302] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce8654d247546adbc74956f8ed0cc75, cf67b74d72c8425390d2cfe9aafed130; no voters:
I20260619 14:59:37.154613 25370 raft_consensus.cc:2781] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:59:37.154830 25370 raft_consensus.cc:482] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:59:37.154935 25370 raft_consensus.cc:3037] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:37.157152 25370 raft_consensus.cc:504] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 } } peers { permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 } } peers { permanent_uuid: "cf67b74d72c8425390d2cfe9aafed130" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 45597 } }
I20260619 14:59:37.157876 25370 leader_election.cc:288] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [CANDIDATE]: Term 1 election: Requested vote from peers a36f76697106467ba5ae1f88de87c85b (127.11.175.193:38741), 2ce8654d247546adbc74956f8ed0cc75 (127.11.175.194:33775)
I20260619 14:59:37.158290 25016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe50b0972f9540fa90e084377dd951df" candidate_uuid: "cf67b74d72c8425390d2cfe9aafed130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a36f76697106467ba5ae1f88de87c85b"
I20260619 14:59:37.158303 25160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe50b0972f9540fa90e084377dd951df" candidate_uuid: "cf67b74d72c8425390d2cfe9aafed130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce8654d247546adbc74956f8ed0cc75"
I20260619 14:59:37.158475 25016 raft_consensus.cc:3037] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:37.158494 25160 raft_consensus.cc:3037] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:37.160841 25160 raft_consensus.cc:2454] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf67b74d72c8425390d2cfe9aafed130 in term 1.
I20260619 14:59:37.161240 25229 leader_election.cc:302] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce8654d247546adbc74956f8ed0cc75, cf67b74d72c8425390d2cfe9aafed130; no voters:
I20260619 14:59:37.161386 25016 raft_consensus.cc:2454] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf67b74d72c8425390d2cfe9aafed130 in term 1.
I20260619 14:59:37.161499 25370 raft_consensus.cc:2781] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:59:37.162060 25370 raft_consensus.cc:686] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 1 LEADER]: Becoming Leader. State: Replica: cf67b74d72c8425390d2cfe9aafed130, State: Running, Role: LEADER
I20260619 14:59:37.162830 25370 consensus_queue.cc:237] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 } } peers { permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 } } peers { permanent_uuid: "cf67b74d72c8425390d2cfe9aafed130" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 45597 } }
I20260619 14:59:37.167330 24863 catalog_manager.cc:5495] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf67b74d72c8425390d2cfe9aafed130 (127.11.175.195). New cstate: current_term: 1 leader_uuid: "cf67b74d72c8425390d2cfe9aafed130" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf67b74d72c8425390d2cfe9aafed130" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 45597 } health_report { overall_health: HEALTHY } } }
I20260619 14:59:37.214195 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:37.208584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57430 (local address 127.11.175.195:45597)
0619 14:59:37.208791 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:37.208796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:37.209124 (+ 328us) server_negotiation.cc:407] Connection header received
0619 14:59:37.209301 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:37.209304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:37.209388 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:37.209519 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:59:37.209873 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.210741 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.212223 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.212618 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.212713 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:37.213196 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:59:37.213708 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:59:37.213824 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:37.213929 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 14:59:37.213998 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 14:59:37.222870 25016 raft_consensus.cc:1261] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b [term 1 FOLLOWER]: Refusing update from remote peer cf67b74d72c8425390d2cfe9aafed130: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:59:37.222898 25160 raft_consensus.cc:1261] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75 [term 1 FOLLOWER]: Refusing update from remote peer cf67b74d72c8425390d2cfe9aafed130: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:59:37.224160 25379 consensus_queue.cc:1035] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ce8654d247546adbc74956f8ed0cc75" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 33775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:59:37.224542 25370 consensus_queue.cc:1035] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a36f76697106467ba5ae1f88de87c85b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:59:37.236132 25383 mvcc.cc:205] Tried to move back new op lower bound from 7298585301893898240 to 7298585301663031296. Current Snapshot: MvccSnapshot[applied={T|T < 7298585301893898240}]
I20260619 14:59:37.258291 24863 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:59:37.258567 24863 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:59:37.262472 24863 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 460da7dff1754e7e8c5a18c885ef92df: Sending DeleteTablet for 3 replicas of tablet fe50b0972f9540fa90e084377dd951df
I20260619 14:59:37.263782 24996 tablet_service.cc:1514] Processing DeleteTablet for tablet fe50b0972f9540fa90e084377dd951df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:59:37 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:34396
I20260619 14:59:37.263943 25274 tablet_service.cc:1514] Processing DeleteTablet for tablet fe50b0972f9540fa90e084377dd951df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:59:37 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:57414
I20260619 14:59:37.264835 25140 tablet_service.cc:1514] Processing DeleteTablet for tablet fe50b0972f9540fa90e084377dd951df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:59:37 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:51054
I20260619 14:59:37.265070 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 24901
I20260619 14:59:37.265324 25394 tablet_replica.cc:330] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: stopping tablet replica
I20260619 14:59:37.267299 25393 tablet_replica.cc:330] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: stopping tablet replica
I20260619 14:59:37.267824 25393 raft_consensus.cc:2229] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:59:37.268633 25393 raft_consensus.cc:2258] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:59:37.269630 25394 raft_consensus.cc:2229] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:59:37.270257 25394 raft_consensus.cc:2258] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:59:37.271282 25395 tablet_replica.cc:330] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: stopping tablet replica
I20260619 14:59:37.271798 25395 raft_consensus.cc:2229] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:59:37.272254 25393 ts_tablet_manager.cc:1905] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:59:37.272382 25395 raft_consensus.cc:2258] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:59:37.274850 25395 ts_tablet_manager.cc:1905] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:59:37.275061 25394 ts_tablet_manager.cc:1905] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:59:37.281790 25395 ts_tablet_manager.cc:1918] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:59:37.281795 25393 ts_tablet_manager.cc:1918] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:59:37.281988 25393 log.cc:1196] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/fe50b0972f9540fa90e084377dd951df
I20260619 14:59:37.281982 25395 log.cc:1196] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/fe50b0972f9540fa90e084377dd951df
I20260619 14:59:37.282545 25393 ts_tablet_manager.cc:1939] T fe50b0972f9540fa90e084377dd951df P cf67b74d72c8425390d2cfe9aafed130: Deleting consensus metadata
I20260619 14:59:37.283031 25395 ts_tablet_manager.cc:1939] T fe50b0972f9540fa90e084377dd951df P 2ce8654d247546adbc74956f8ed0cc75: Deleting consensus metadata
I20260619 14:59:37.284576 24849 catalog_manager.cc:4841] TS cf67b74d72c8425390d2cfe9aafed130 (127.11.175.195:45597): tablet fe50b0972f9540fa90e084377dd951df (table test-table [id=fc4ff75d8285461fb1b27afc99ff62ac]) successfully deleted
I20260619 14:59:37.285018 24850 catalog_manager.cc:4841] TS 2ce8654d247546adbc74956f8ed0cc75 (127.11.175.194:33775): tablet fe50b0972f9540fa90e084377dd951df (table test-table [id=fc4ff75d8285461fb1b27afc99ff62ac]) successfully deleted
I20260619 14:59:37.286980 25394 ts_tablet_manager.cc:1918] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:59:37.287242 25394 log.cc:1196] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/fe50b0972f9540fa90e084377dd951df
I20260619 14:59:37.287768 25394 ts_tablet_manager.cc:1939] T fe50b0972f9540fa90e084377dd951df P a36f76697106467ba5ae1f88de87c85b: Deleting consensus metadata
I20260619 14:59:37.289608 24851 catalog_manager.cc:4841] TS a36f76697106467ba5ae1f88de87c85b (127.11.175.193:38741): tablet fe50b0972f9540fa90e084377dd951df (table test-table [id=fc4ff75d8285461fb1b27afc99ff62ac]) successfully deleted
I20260619 14:59:37.295634 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:37.265955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.11.175.193:38741)
0619 14:59:37.266133 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:37.266140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:37.273271 (+ 7131us) server_negotiation.cc:407] Connection header received
0619 14:59:37.273909 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:37.273916 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:37.274011 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:37.274141 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:37.276713 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.277893 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.280045 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.280713 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.280830 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:37.286917 (+ 6087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:37.286935 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:37.286955 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:37.287020 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:37.291900 (+ 4880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:37.292951 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:37.292962 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:37.292966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:37.293139 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:37.293692 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:37.293698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:37.293701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:37.294183 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:37.294516 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:37.294979 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 14:59:37.295228 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 14:59:37.296494 25056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:37.387853 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 24901
I20260619 14:59:37.405525 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 25065
I20260619 14:59:37.420548 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:37.406363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.11.175.194:33775)
0619 14:59:37.406552 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:37.406558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:37.406702 (+ 144us) server_negotiation.cc:407] Connection header received
0619 14:59:37.406963 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:37.406968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:37.407054 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:37.407169 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:37.408542 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.409244 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.410588 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.411063 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.411202 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:37.413857 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:37.413881 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:37.413898 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:37.413934 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:37.417263 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:37.418043 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:37.418051 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:37.418053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:37.418198 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:37.418701 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:37.418705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:37.418709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:37.419149 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:37.419462 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:37.419881 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 14:59:37.420120 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 14:59:37.421211 25200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:37.510782 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 25065
I20260619 14:59:37.528502 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 25209
I20260619 14:59:37.546713 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:37.529300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57444 (local address 127.11.175.195:45597)
0619 14:59:37.529471 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:37.529475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:37.529551 (+ 76us) server_negotiation.cc:407] Connection header received
0619 14:59:37.529742 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:37.529746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:37.529843 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:37.529931 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:37.531836 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.532593 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.534417 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:37.535079 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:37.535183 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:37.538766 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:37.538815 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:37.538836 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:37.538894 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:37.542763 (+ 3869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:37.544022 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:37.544030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:37.544032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:37.544223 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:37.544663 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:37.544667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:37.544669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:37.545094 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:37.545329 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:37.545814 (+ 485us) server_negotiation.cc:299] Negotiation successful
0619 14:59:37.546073 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 14:59:37.547582 25335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:37.670828 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 25209
I20260619 14:59:37.690698 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 24831
I20260619 14:59:37.692013 24894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:37.805609 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 24831
I20260619 14:59:37.830463 11967 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 14:59:38.166128 11967 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 14:59:38.166450 11967 mini_ranger.cc:66] Stopping Ranger...
I20260619 14:59:38.512439 11967 mini_ranger.cc:68] Stopped Ranger
2026-06-19 14:59:38.512 UTC [23883] LOG: received smart shutdown request
2026-06-19 14:59:38.515 UTC [23883] LOG: background worker "logical replication launcher" (PID 23892) exited with exit code 1
2026-06-19 14:59:38.515 UTC [23887] LOG: shutting down
2026-06-19 14:59:38.690 UTC [23883] LOG: database system is shut down
2026-06-19T14:59:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53992 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:59:38 dist-test-slave-76rk krb5kdc[25409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:59:38 dist-test-slave-76rk krb5kdc[25409](info): set up 2 sockets
Jun 19 14:59:38 dist-test-slave-76rk krb5kdc[25409](info): commencing operation
krb5kdc: starting...
W20260619 14:59:39.947232 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:59:40 dist-test-slave-76rk krb5kdc[25409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:59:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:59:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:59:40.202033 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38871
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38871,127.11.175.253:43425,127.11.175.252:37249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:59:40.541898 25422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:40.542182 25422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:40.542354 25422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:40.549815 25422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:59:40.549932 25422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:40.550017 25422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:40.550127 25422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:59:40.550218 25422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:59:40.560195 25422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38871,127.11.175.253:43425,127.11.175.252:37249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:40.561605 25422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:40.563032 25422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:40.575596 25427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:40.575682 25430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:40.575656 25428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:40.576864 25422 server_base.cc:1034] running on GCE node
I20260619 14:59:40.577893 25422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:40.579708 25422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:40.580996 25422 hybrid_clock.cc:648] HybridClock initialized: now 1781881180580942 us; error 53 us; skew 500 ppm
I20260619 14:59:40.581423 25422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:40 dist-test-slave-76rk krb5kdc[25409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881180, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:40.591648 25422 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:59:40.593367 25422 webserver.cc:460] Webserver started at http://127.11.175.254:41339/ using document root <none> and password file <none>
I20260619 14:59:40.594117 25422 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:40.594242 25422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:40.594509 25422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:59:40.597759 25422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "ef5087f0e9284f0bb6c0a3378f530766"
format_stamp: "Formatted at 2026-06-19 14:59:40 on dist-test-slave-76rk"
server_key: "fbe0a081acdf676d5d5832b420b8face"
server_key_iv: "95f88f72691e86c31f243dcdb7920c8d"
server_key_version: "encryptionkey@0"
I20260619 14:59:40.598526 25422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "ef5087f0e9284f0bb6c0a3378f530766"
format_stamp: "Formatted at 2026-06-19 14:59:40 on dist-test-slave-76rk"
server_key: "fbe0a081acdf676d5d5832b420b8face"
server_key_iv: "95f88f72691e86c31f243dcdb7920c8d"
server_key_version: "encryptionkey@0"
I20260619 14:59:40.603914 25422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 14:59:40.607651 25437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:40.609467 25422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:59:40.609741 25422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "ef5087f0e9284f0bb6c0a3378f530766"
format_stamp: "Formatted at 2026-06-19 14:59:40 on dist-test-slave-76rk"
server_key: "fbe0a081acdf676d5d5832b420b8face"
server_key_iv: "95f88f72691e86c31f243dcdb7920c8d"
server_key_version: "encryptionkey@0"
I20260619 14:59:40.609982 25422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:40.631983 25422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:40.636582 25422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:40.636937 25422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:40.656217 25422 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:38871
I20260619 14:59:40.656265 25488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:38871 every 8 connection(s)
I20260619 14:59:40.657711 25422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:59:40.661623 25489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:40.667055 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 25422
I20260619 14:59:40.667285 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:59:40.668026 11967 external_mini_cluster.cc:1428] Setting key d1ca8aab86f54d477772189e0a92d0e4
I20260619 14:59:40.668591 25489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:59:40 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881180, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.175.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.253@KRBTEST.COM" created.
I20260619 14:59:40.695351 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:40.672086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38871 (local address 127.0.0.1:34774)
0619 14:59:40.673855 (+ 1769us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:40.673909 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:40.676786 (+ 2877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:40.677722 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:40.677730 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:40.678497 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:40.678928 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:40.678949 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:40.680486 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:40.680491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:40.681600 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:40.681612 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:40.682562 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:40.682569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:40.682789 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:40.683477 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:40.683499 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:40.686068 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:40.690813 (+ 4745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:40.690824 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:40.690840 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:40.691351 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:40.691867 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:40.691871 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:40.691873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:40.692052 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:40.692998 (+ 946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:40.693004 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:40.693258 (+ 254us) client_negotiation.cc:769] Sending connection context
0619 14:59:40.693522 (+ 264us) client_negotiation.cc:240] Negotiation successful
0619 14:59:40.694149 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1441,"thread_start_us":1198,"threads_started":1}
I20260619 14:59:40.695400 25492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:40.672086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.11.175.254:38871)
0619 14:59:40.675065 (+ 2979us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:40.675072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:40.675119 (+ 47us) server_negotiation.cc:407] Connection header received
0619 14:59:40.677225 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:40.677231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:40.677481 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:40.677711 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:40.679132 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:40.680324 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:40.681745 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:40.682329 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:40.682546 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:40.686334 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:40.686342 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:40.686355 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:40.686389 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:40.690549 (+ 4160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:40.691501 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:40.691508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:40.691511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:40.691670 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:40.692217 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:40.692222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:40.692226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:40.692824 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:40.693066 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:40.693824 (+ 758us) server_negotiation.cc:299] Negotiation successful
0619 14:59:40.694407 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":187,"server-negotiator.queue_time_us":2770,"spinlock_wait_cycles":26880,"thread_start_us":2523,"threads_started":1}
I20260619 14:59:40.698559 25489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } has no permanent_uuid. Determining permanent_uuid...
I20260619 14:59:40.699834 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:40.699437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39380)
0619 14:59:40.699599 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:40.699708 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260619 14:59:40.700235 25439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
I20260619 14:59:40.701761 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:40.701436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39390)
0619 14:59:40.701568 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:40.701644 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 14:59:40.702280 25489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:43425: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab.
Entry for principal kudu/127.11.175.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab.
I20260619 14:59:40.724097 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.253:43425
--webserver_interface=127.11.175.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab
--principal=kudu/127.11.175.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38871,127.11.175.253:43425,127.11.175.252:37249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:59:40.751658 25489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } attempt: 1
I20260619 14:59:40.752903 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:40.752423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39404)
0619 14:59:40.752626 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:40.752759 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 14:59:40.753895 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:40.753578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39416)
0619 14:59:40.753707 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:40.753773 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 14:59:40.754112 25489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:43425: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
I20260619 14:59:40.822383 25489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } attempt: 2
I20260619 14:59:40.823619 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:40.823147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39418)
0619 14:59:40.823388 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:40.823479 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 14:59:40.824704 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:40.824342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39434)
0619 14:59:40.824500 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:40.824580 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 14:59:40.824975 25489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:43425: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
I20260619 14:59:40.916235 25489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } attempt: 3
I20260619 14:59:40.917675 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:40.917086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39444)
0619 14:59:40.917351 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:40.917479 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260619 14:59:40.918991 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:40.918452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39454)
0619 14:59:40.918675 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:40.918799 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260619 14:59:40.919227 25489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:43425: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
I20260619 14:59:41.062510 25489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } attempt: 4
I20260619 14:59:41.063898 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.063260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39468)
0619 14:59:41.063613 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.063729 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260619 14:59:41.065085 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.064659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39472)
0619 14:59:41.064818 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.064931 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 14:59:41.065368 25489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:43425: Network error: Client connection negotiation failed: client connection to 127.11.175.253:43425: connect: Connection refused (error 111)
W20260619 14:59:41.067049 25496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:41.067351 25496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:41.067518 25496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:41.075103 25496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:59:41.075223 25496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:41.075306 25496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:41.075379 25496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:59:41.075462 25496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:59:41.085645 25496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38871,127.11.175.253:43425,127.11.175.252:37249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab
--principal=kudu/127.11.175.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.253:43425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.175.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:41.087226 25496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:41.088593 25496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:41.100728 25502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:41.100805 25501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:41.101301 25504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:41.101545 25496 server_base.cc:1034] running on GCE node
I20260619 14:59:41.102376 25496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:41.103819 25496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:41.105044 25496 hybrid_clock.cc:648] HybridClock initialized: now 1781881181104973 us; error 58 us; skew 500 ppm
I20260619 14:59:41.105419 25496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881181, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:41.108722 25496 init.cc:377] Logged in from keytab as kudu/127.11.175.253@KRBTEST.COM (short username kudu)
I20260619 14:59:41.110271 25496 webserver.cc:460] Webserver started at http://127.11.175.253:42187/ using document root <none> and password file <none>
I20260619 14:59:41.111052 25496 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:41.111202 25496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:41.111518 25496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:59:41.114670 25496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/data/instance:
uuid: "258280603d46496ebc5d7b3c8a34ed77"
format_stamp: "Formatted at 2026-06-19 14:59:41 on dist-test-slave-76rk"
server_key: "3f7b860ebc91d5baf0a3978435b7f52f"
server_key_iv: "475b3d999a4074cc575f966da38b90fb"
server_key_version: "encryptionkey@0"
I20260619 14:59:41.115513 25496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/wal/instance:
uuid: "258280603d46496ebc5d7b3c8a34ed77"
format_stamp: "Formatted at 2026-06-19 14:59:41 on dist-test-slave-76rk"
server_key: "3f7b860ebc91d5baf0a3978435b7f52f"
server_key_iv: "475b3d999a4074cc575f966da38b90fb"
server_key_version: "encryptionkey@0"
I20260619 14:59:41.120980 25496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260619 14:59:41.125190 25511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:41.126772 25496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:59:41.126963 25496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/wal
uuid: "258280603d46496ebc5d7b3c8a34ed77"
format_stamp: "Formatted at 2026-06-19 14:59:41 on dist-test-slave-76rk"
server_key: "3f7b860ebc91d5baf0a3978435b7f52f"
server_key_iv: "475b3d999a4074cc575f966da38b90fb"
server_key_version: "encryptionkey@0"
I20260619 14:59:41.127161 25496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:41.156719 25496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:41.161427 25496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:41.161687 25496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:41.182403 25496 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.253:43425
I20260619 14:59:41.182411 25562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.253:43425 every 8 connection(s)
I20260619 14:59:41.183905 25496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
I20260619 14:59:41.187703 25563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:41.188488 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 25496
I20260619 14:59:41.188688 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-1/wal/instance
I20260619 14:59:41.189254 11967 external_mini_cluster.cc:1428] Setting key 1551ac2496bbff90da89bdae1f9ddf05
I20260619 14:59:41.198333 25563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881181, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.175.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.252@KRBTEST.COM" created.
I20260619 14:59:41.221536 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.201332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.11.175.254:38871)
0619 14:59:41.201824 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.201834 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.203152 (+ 1318us) server_negotiation.cc:407] Connection header received
0619 14:59:41.205232 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.205240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.205343 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.205471 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.207240 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.207998 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.209642 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.210381 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.210558 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.214613 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.214618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.214622 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.214718 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.217935 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.219067 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.219071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.219074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.219196 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.219703 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.219706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.219708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.219981 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.220106 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.221096 (+ 990us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.221275 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":281,"threads_started":1}
I20260619 14:59:41.223131 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.202034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38871 (local address 127.0.0.1:34782)
0619 14:59:41.202975 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.203021 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:41.204901 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:41.205752 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:41.205762 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:41.206322 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:41.207033 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.207052 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.208193 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.208197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.209475 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.209489 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.210716 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.210728 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.210968 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.211728 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:41.211754 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:41.214354 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:41.218122 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.218135 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.218158 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.218924 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.219308 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.219315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.219322 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.219590 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.220113 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:41.220124 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:41.220656 (+ 532us) client_negotiation.cc:769] Sending connection context
0619 14:59:41.220995 (+ 339us) client_negotiation.cc:240] Negotiation successful
0619 14:59:41.221582 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":388,"threads_started":1}
I20260619 14:59:41.225008 25563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881181, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 14:59:41.240814 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.225839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39478)
0619 14:59:41.226017 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.226038 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:41.226147 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:41.227413 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:41.227417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:41.227863 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:41.227990 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.227998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.229309 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.229316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.230143 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.230154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.231140 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.231145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.231371 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.232066 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:41.232099 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:41.234860 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:41.238152 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.238157 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.238161 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.238542 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.238962 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.238967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.238970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.239116 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.239839 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:41.239844 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:41.239982 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 14:59:41.240124 (+ 142us) client_negotiation.cc:240] Negotiation successful
0619 14:59:41.240420 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 14:59:41.241025 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.226145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.11.175.253:43425)
0619 14:59:41.226998 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.227004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.227030 (+ 26us) server_negotiation.cc:407] Connection header received
0619 14:59:41.227077 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.227082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.227257 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.227429 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.228137 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.229163 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.230276 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.231005 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.231134 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.235071 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.235082 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.235095 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.235122 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.238005 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.238704 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.238713 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.238717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.238845 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.239216 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.239223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.239226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.239686 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.239909 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.240325 (+ 416us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.240555 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":449,"threads_started":1}
I20260619 14:59:41.243981 25563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } has no permanent_uuid. Determining permanent_uuid...
I20260619 14:59:41.245476 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.244938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:58990)
0619 14:59:41.245126 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.245266 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260619 14:59:41.245759 25514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab.
Entry for principal kudu/127.11.175.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab.
I20260619 14:59:41.247510 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.247067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:58992)
0619 14:59:41.247238 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.247361 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 14:59:41.247756 25563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:37249: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
I20260619 14:59:41.248337 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.252:37249
--webserver_interface=127.11.175.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab
--principal=kudu/127.11.175.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38871,127.11.175.253:43425,127.11.175.252:37249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:59:41.297098 25563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } attempt: 1
I20260619 14:59:41.298425 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.297908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:59008)
0619 14:59:41.298148 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.298263 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260619 14:59:41.299748 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.299310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:59014)
0619 14:59:41.299495 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.299592 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 14:59:41.300015 25563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:37249: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
I20260619 14:59:41.364703 25489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } attempt: 5
I20260619 14:59:41.368253 25563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } attempt: 2
I20260619 14:59:41.369264 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.368802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:59030)
0619 14:59:41.368971 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.369071 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 14:59:41.370396 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.369941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:59040)
0619 14:59:41.370113 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.370208 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260619 14:59:41.370682 25563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:37249: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881180, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 14:59:41.384004 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.365720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39490 (local address 127.11.175.253:43425)
0619 14:59:41.365918 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.365926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.365951 (+ 25us) server_negotiation.cc:407] Connection header received
0619 14:59:41.366024 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.366028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.366105 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.366195 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.367319 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.368026 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.369689 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.370321 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.371119 (+ 798us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.378678 (+ 7559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.378691 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.378695 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.378730 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.381281 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.382011 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.382017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.382021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.382149 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.382540 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.382546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.382549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.382909 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.383065 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.383448 (+ 383us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.383673 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 14:59:41.384030 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.365498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:39490)
0619 14:59:41.365733 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.365758 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:41.365909 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:41.366265 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:41.366270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:41.366858 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:41.367090 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.367103 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.368186 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.368191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.369520 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.369535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.370538 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.370548 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.370823 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.375833 (+ 5010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:41.375856 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:41.378400 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:41.381477 (+ 3077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.381482 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.381487 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.381880 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.382269 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.382274 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.382278 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.382425 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.383031 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:41.383036 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:41.383194 (+ 158us) client_negotiation.cc:769] Sending connection context
0619 14:59:41.383350 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 14:59:41.383675 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 14:59:41.385226 25489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } has no permanent_uuid. Determining permanent_uuid...
I20260619 14:59:41.386353 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.385936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:59048)
0619 14:59:41.386098 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.386177 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 14:59:41.387351 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.387008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:59054)
0619 14:59:41.387151 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.387244 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 14:59:41.387543 25489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:37249: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
I20260619 14:59:41.438830 25489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } attempt: 1
I20260619 14:59:41.440052 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.439570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36678)
0619 14:59:41.439764 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.439889 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 14:59:41.441162 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.440788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36690)
0619 14:59:41.440973 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.441054 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260619 14:59:41.441381 25489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:37249: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
I20260619 14:59:41.461930 25563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } attempt: 3
I20260619 14:59:41.463315 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.462813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36692)
0619 14:59:41.463027 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.463141 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 14:59:41.464475 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.464038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36708)
0619 14:59:41.464272 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.464355 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260619 14:59:41.464710 25563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:37249: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
I20260619 14:59:41.509585 25489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } attempt: 2
I20260619 14:59:41.510852 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.510320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36718)
0619 14:59:41.510541 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.510660 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 14:59:41.511947 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.511599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36730)
0619 14:59:41.511753 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.511827 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 14:59:41.512148 25489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:37249: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
W20260619 14:59:41.604597 25571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:41.604957 25571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:41.605121 25571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 14:59:41.607913 25563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } attempt: 4
I20260619 14:59:41.609062 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.608588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36734)
0619 14:59:41.608811 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.608907 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 14:59:41.610018 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.609710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36744)
0619 14:59:41.609848 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.609921 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 14:59:41.610213 25563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:37249: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
W20260619 14:59:41.612969 25571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:59:41.613111 25571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:41.613209 25571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:41.613327 25571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:59:41.613401 25571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:59:41.618391 25489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } attempt: 3
I20260619 14:59:41.619442 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.619029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36746)
0619 14:59:41.619205 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.619288 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 14:59:41.620458 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.620082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36754)
0619 14:59:41.620290 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.620362 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260619 14:59:41.620693 25489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:37249: Network error: Client connection negotiation failed: client connection to 127.11.175.252:37249: connect: Connection refused (error 111)
I20260619 14:59:41.624204 25571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38871,127.11.175.253:43425,127.11.175.252:37249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab
--principal=kudu/127.11.175.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.252:37249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.175.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:41.625552 25571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:41.626963 25571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:41.638746 25577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:41.638826 25576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:41.638862 25579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:41.640075 25571 server_base.cc:1034] running on GCE node
I20260619 14:59:41.640889 25571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:41.642373 25571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:41.643573 25571 hybrid_clock.cc:648] HybridClock initialized: now 1781881181643491 us; error 81 us; skew 500 ppm
I20260619 14:59:41.643879 25571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881181, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:41.647136 25571 init.cc:377] Logged in from keytab as kudu/127.11.175.252@KRBTEST.COM (short username kudu)
I20260619 14:59:41.648604 25571 webserver.cc:460] Webserver started at http://127.11.175.252:37363/ using document root <none> and password file <none>
I20260619 14:59:41.649277 25571 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:41.649382 25571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:41.649628 25571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:59:41.652593 25571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/data/instance:
uuid: "ea4967da854e47708edc1f4be95d6255"
format_stamp: "Formatted at 2026-06-19 14:59:41 on dist-test-slave-76rk"
server_key: "0c9df4b0c313607f1cabbacc1193ac44"
server_key_iv: "2b3660a20519a5f96885b31aa216de8d"
server_key_version: "encryptionkey@0"
I20260619 14:59:41.653316 25571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/wal/instance:
uuid: "ea4967da854e47708edc1f4be95d6255"
format_stamp: "Formatted at 2026-06-19 14:59:41 on dist-test-slave-76rk"
server_key: "0c9df4b0c313607f1cabbacc1193ac44"
server_key_iv: "2b3660a20519a5f96885b31aa216de8d"
server_key_version: "encryptionkey@0"
I20260619 14:59:41.658835 25571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260619 14:59:41.662494 25586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:41.664098 25571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:59:41.664286 25571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/wal
uuid: "ea4967da854e47708edc1f4be95d6255"
format_stamp: "Formatted at 2026-06-19 14:59:41 on dist-test-slave-76rk"
server_key: "0c9df4b0c313607f1cabbacc1193ac44"
server_key_iv: "2b3660a20519a5f96885b31aa216de8d"
server_key_version: "encryptionkey@0"
I20260619 14:59:41.664513 25571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:41.685199 25571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:41.689447 25571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:41.689744 25571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:41.709342 25571 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.252:37249
I20260619 14:59:41.709374 25637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.252:37249 every 8 connection(s)
I20260619 14:59:41.710781 25571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
I20260619 14:59:41.712507 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 25571
I20260619 14:59:41.712721 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781880972997616-11967-0/minicluster-data/master-2/wal/instance
I20260619 14:59:41.713414 11967 external_mini_cluster.cc:1428] Setting key 26b7de9ae9394a55368190e63bb9866e
I20260619 14:59:41.716048 25638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:41.724826 25638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881181, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:41.737697 25641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.715585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36760 (local address 127.11.175.252:37249)
0619 14:59:41.716434 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.716447 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.716487 (+ 40us) server_negotiation.cc:407] Connection header received
0619 14:59:41.718112 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.718133 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.718526 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.718948 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.720779 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.722163 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.724185 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.724875 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.725018 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.730074 (+ 5056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.730082 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.730092 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.730132 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.733466 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.734198 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.734212 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.734219 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.734389 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.734847 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.734851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.734853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.735376 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.735726 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.736163 (+ 437us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.736621 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":402,"threads_started":1}
I20260619 14:59:41.743649 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.728234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.11.175.254:38871)
0619 14:59:41.728741 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.728750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.729284 (+ 534us) server_negotiation.cc:407] Connection header received
0619 14:59:41.729519 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.729523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.729596 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.729724 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.731002 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.732155 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.733662 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.734189 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.734383 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.737673 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.737678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.737681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.737723 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.740804 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.741496 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.741503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.741506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.741694 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.742043 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.742047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.742050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.742451 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.742574 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.743218 (+ 644us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.743377 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":250,"threads_started":1}
I20260619 14:59:41.743885 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.728261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38871 (local address 127.0.0.1:48458)
0619 14:59:41.729147 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.729192 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:41.729378 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:41.729790 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:41.729796 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:41.730451 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:41.730823 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.730839 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.732322 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.732326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.733502 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.733518 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.734417 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.734424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.734624 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.735340 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:41.735358 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:41.737458 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:41.740952 (+ 3494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.740963 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.740971 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.741374 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.741814 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.741817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.741820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.741942 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.742593 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:41.742603 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:41.742859 (+ 256us) client_negotiation.cc:769] Sending connection context
0619 14:59:41.743103 (+ 244us) client_negotiation.cc:240] Negotiation successful
0619 14:59:41.743455 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":489,"threads_started":1}
I20260619 14:59:41.745602 25638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881181, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 14:59:41.758386 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.740699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.11.175.254:38871)
0619 14:59:41.741962 (+ 1263us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.741966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.741983 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:59:41.742048 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.742052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.742136 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.742243 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.743851 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.745069 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.746737 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.747352 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.747444 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.752576 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.752582 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.752585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.752620 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.755544 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.756284 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.756288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.756290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.756430 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.756855 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.756866 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.756868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.757145 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.757451 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.757727 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.757963 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1124,"thread_start_us":888,"threads_started":1}
I20260619 14:59:41.760138 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.746434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:43425 (local address 127.0.0.1:54248)
0619 14:59:41.746597 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.746622 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:41.746762 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:41.747193 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:41.747197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:41.747721 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:41.747946 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.747962 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.749047 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.749054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.750566 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.750584 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.751328 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.751334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.751527 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.752102 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:41.752118 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:41.754419 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:41.757428 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.757434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.757439 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.757826 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.758274 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.758280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.758283 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.758475 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.759164 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:41.759170 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:41.759340 (+ 170us) client_negotiation.cc:769] Sending connection context
0619 14:59:41.759486 (+ 146us) client_negotiation.cc:240] Negotiation successful
0619 14:59:41.759780 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 14:59:41.760205 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.746439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.11.175.253:43425)
0619 14:59:41.746630 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.746655 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.746865 (+ 210us) server_negotiation.cc:407] Connection header received
0619 14:59:41.746926 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.746931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.747013 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.747110 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.748137 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.748884 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.750744 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.751149 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.751294 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.754618 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.754623 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.754626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.754713 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.757275 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.757956 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.757962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.757966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.758142 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.758684 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.758692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.758696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.759030 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.759198 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.759582 (+ 384us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.759819 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 14:59:41.761343 25638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881181, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 14:59:41.777709 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.762057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36766)
0619 14:59:41.762310 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.762324 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:41.762395 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:41.762714 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:41.762718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:41.763177 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:41.763342 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.763356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.765056 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.765062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.766117 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.766129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.767019 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.767028 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.767312 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.767970 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:41.768002 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:41.771295 (+ 3293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:41.774892 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.774898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.774903 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.775441 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.775816 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.775824 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.775829 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.775989 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.776667 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:41.776673 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:41.776840 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 14:59:41.777009 (+ 169us) client_negotiation.cc:240] Negotiation successful
0619 14:59:41.777313 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260619 14:59:41.777767 25641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.762133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36766 (local address 127.11.175.252:37249)
0619 14:59:41.762336 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.762344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.762369 (+ 25us) server_negotiation.cc:407] Connection header received
0619 14:59:41.762510 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.762515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.762575 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.762686 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.763518 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.764897 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.766258 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.766828 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.766952 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.771541 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.771548 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.771554 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.771596 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.774736 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.775561 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.775564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.775567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.775687 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.776099 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.776103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.776107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.776519 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.776671 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.777128 (+ 457us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.777494 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":41}
I20260619 14:59:41.779125 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.760185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.11.175.253:43425)
0619 14:59:41.760521 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.760529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.760548 (+ 19us) server_negotiation.cc:407] Connection header received
0619 14:59:41.760599 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.760603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.760673 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.760770 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.763063 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.764011 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.765505 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.766411 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.766560 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.772424 (+ 5864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.772434 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.772442 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.772494 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.775759 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.776746 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.776753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.776757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.776920 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.777347 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.777351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.777354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.777720 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.777994 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.778478 (+ 484us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.778739 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":12535,"threads_started":1}
I20260619 14:59:41.783826 25638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255: Bootstrap starting.
I20260619 14:59:41.784248 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 14:59:41.787324 25638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:41.788489 25638 log.cc:824] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:41.791752 25638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255: No bootstrap required, opened a new log
I20260619 14:59:41.797880 25489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } attempt: 4
I20260619 14:59:41.812538 25638 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } }
I20260619 14:59:41.817123 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.790853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54264 (local address 127.11.175.253:43425)
0619 14:59:41.791519 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.791525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.792457 (+ 932us) server_negotiation.cc:407] Connection header received
0619 14:59:41.792618 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.792625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.792704 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.792823 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.795513 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.796957 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.799698 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.800586 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.800738 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.805886 (+ 5148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.805897 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.805905 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.805967 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.811344 (+ 5377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.812645 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.812654 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.812660 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.812914 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.813535 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.813543 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.813548 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.814053 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.814453 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.816232 (+ 1779us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.816593 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541}
I20260619 14:59:41.816031 25641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.791112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.11.175.252:37249)
0619 14:59:41.794797 (+ 3685us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.794801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.794818 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:59:41.794859 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.794862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.794913 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.795028 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.797123 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.798179 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.800863 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.801877 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.802030 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.806275 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.806288 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.806292 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.806336 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.810926 (+ 4590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.811996 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.812004 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.812011 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.812197 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.812757 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.812764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.812772 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.813322 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.813668 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.815145 (+ 1477us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.815466 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3567}
I20260619 14:59:41.817878 25638 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:41.818104 25638 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea4967da854e47708edc1f4be95d6255, State: Initialized, Role: FOLLOWER
I20260619 14:59:41.819453 25638 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } }
I20260619 14:59:41.821257 25655 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } } }
I20260619 14:59:41.821816 25655 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:59:41.822785 25638 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [sys.catalog]: configured and running, proceeding with master startup.
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881180, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
I20260619 14:59:41.835618 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.798453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36782)
0619 14:59:41.798839 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.798859 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:41.798968 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:41.816931 (+ 17963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:41.816938 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:41.817465 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:41.817741 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.817755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.821270 (+ 3515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.821276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.822701 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.822720 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.823764 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.823769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.823983 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.824721 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:41.824746 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:41.828051 (+ 3305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:41.832484 (+ 4433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.832490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.832494 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.833056 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.833560 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.833565 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.833569 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.833727 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.834488 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:41.834493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:41.834702 (+ 209us) client_negotiation.cc:769] Sending connection context
0619 14:59:41.834872 (+ 170us) client_negotiation.cc:240] Negotiation successful
0619 14:59:41.835177 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261}
I20260619 14:59:41.835863 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.790599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48470 (local address 127.11.175.254:38871)
0619 14:59:41.790854 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.790861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.815991 (+ 25130us) server_negotiation.cc:407] Connection header received
0619 14:59:41.817655 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.817669 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.817773 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.817872 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.820172 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.821218 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.822865 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.823287 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.823434 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.827211 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.827222 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.827230 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.827281 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.831481 (+ 4200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.832483 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.832491 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.832498 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.832701 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.833191 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.833199 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.833203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.833584 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.835080 (+ 1496us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.835207 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.835452 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"spinlock_wait_cycles":1792}
I20260619 14:59:41.837347 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.798808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36782 (local address 127.11.175.252:37249)
0619 14:59:41.816615 (+ 17807us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.816619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.816636 (+ 17us) server_negotiation.cc:407] Connection header received
0619 14:59:41.816675 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.816678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.816733 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.816873 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.819582 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.821084 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.822928 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.823592 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.823721 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.828275 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.828282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.828288 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.828351 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.832313 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.833208 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.833213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.833217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.833415 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.833891 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.833900 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.833904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.834330 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.836491 (+ 2161us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.836632 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.836879 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17701,"thread_start_us":18262,"threads_started":1}
W20260619 14:59:41.841290 25666 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 14:59:41.841486 25666 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 14:59:41.831843 25571 master_runner.cc:186] Error getting master registration for 127.11.175.254:38871: OK, instance_id { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" instance_seqno: 1781881180650721 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 14:59:41.843115 25571 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 14:59:41.843643 25489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766: Bootstrap starting.
I20260619 14:59:41.848074 25489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:41.849393 25489 log.cc:824] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:41.853178 25489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766: No bootstrap required, opened a new log
I20260619 14:59:41.864537 25489 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } }
I20260619 14:59:41.865022 25489 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:41.865159 25489 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef5087f0e9284f0bb6c0a3378f530766, State: Initialized, Role: FOLLOWER
I20260619 14:59:41.865876 25489 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } }
I20260619 14:59:41.867002 25669 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } } }
I20260619 14:59:41.868444 25489 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:59:41.868748 25669 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 14:59:41.879312 25680 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 14:59:41.879560 25680 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 14:59:41.877887 25422 master_runner.cc:186] Error getting master registration for 127.11.175.253:43425: OK, instance_id { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" instance_seqno: 1781881181176616 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 14:59:41.889515 25422 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 14:59:41.890061 25443 tablet.cc:2369] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:59:41.908428 25669 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:59:41.908752 25669 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } }
I20260619 14:59:41.909427 25563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } attempt: 5
I20260619 14:59:41.910537 25669 leader_election.cc:288] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 258280603d46496ebc5d7b3c8a34ed77 (127.11.175.253:43425), ea4967da854e47708edc1f4be95d6255 (127.11.175.252:37249)
I20260619 14:59:41.912185 25613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef5087f0e9284f0bb6c0a3378f530766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea4967da854e47708edc1f4be95d6255" is_pre_election: true
I20260619 14:59:41.912850 25613 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef5087f0e9284f0bb6c0a3378f530766 in term 0.
I20260619 14:59:41.913559 25440 leader_election.cc:302] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea4967da854e47708edc1f4be95d6255, ef5087f0e9284f0bb6c0a3378f530766; no voters:
I20260619 14:59:41.914178 25669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:59:41.914333 25669 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:59:41.914445 25669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:41.917949 25669 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } }
Jun 19 14:59:41 dist-test-slave-76rk krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881181, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
I20260619 14:59:41.922750 25613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef5087f0e9284f0bb6c0a3378f530766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea4967da854e47708edc1f4be95d6255"
I20260619 14:59:41.922988 25613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:41.927309 25613 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef5087f0e9284f0bb6c0a3378f530766 in term 1.
I20260619 14:59:41.928122 25440 leader_election.cc:302] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea4967da854e47708edc1f4be95d6255, ef5087f0e9284f0bb6c0a3378f530766; no voters:
I20260619 14:59:41.931576 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.910071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:37249 (local address 127.0.0.1:36790)
0619 14:59:41.910483 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:41.910505 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:41.910624 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:41.911949 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:41.911955 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:41.912464 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:41.912669 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.912682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.914681 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.914688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.916157 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:41.916175 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.917404 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.917410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:41.917651 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.918352 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:41.918378 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:41.923097 (+ 4719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:41.927988 (+ 4891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.927994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.927999 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.928615 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.929089 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:41.929094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:41.929100 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:41.929256 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:41.930015 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:41.930021 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:41.930190 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 14:59:41.930346 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 14:59:41.931167 (+ 821us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279}
I20260619 14:59:41.932116 25669 leader_election.cc:288] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [CANDIDATE]: Term 1 election: Requested vote from peers 258280603d46496ebc5d7b3c8a34ed77 (127.11.175.253:43425), ea4967da854e47708edc1f4be95d6255 (127.11.175.252:37249)
I20260619 14:59:41.932399 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:41.910884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.11.175.252:37249)
0619 14:59:41.911073 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:41.911078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:41.911097 (+ 19us) server_negotiation.cc:407] Connection header received
0619 14:59:41.911174 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:41.911181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:41.911277 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:41.911382 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:41.913151 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.914351 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.916342 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:41.917079 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:41.917204 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:41.923318 (+ 6114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:41.923326 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:41.923331 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:41.923374 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:41.927789 (+ 4415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.928766 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.928773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.928777 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.928953 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:41.929388 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:41.929393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:41.929396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:41.929774 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:41.929933 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:41.930806 (+ 873us) server_negotiation.cc:299] Negotiation successful
0619 14:59:41.931980 (+ 1174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 14:59:41.936722 25669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:59:41.937119 25669 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [term 1 LEADER]: Becoming Leader. State: Replica: ef5087f0e9284f0bb6c0a3378f530766, State: Running, Role: LEADER
I20260619 14:59:41.938388 25669 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } }
I20260619 14:59:41.940347 25563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77: Bootstrap starting.
W20260619 14:59:41.941931 25592 tablet.cc:2369] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:59:41.946135 25563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:41.947769 25563 log.cc:824] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:41.954006 25682 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef5087f0e9284f0bb6c0a3378f530766. Latest consensus state: current_term: 1 leader_uuid: "ef5087f0e9284f0bb6c0a3378f530766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } } }
I20260619 14:59:41.954491 25682 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:41.958170 25685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:59:41.960642 25685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:59:41.965039 25563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77: No bootstrap required, opened a new log
I20260619 14:59:41.995071 25563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } }
I20260619 14:59:41.999420 25563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:41.999644 25563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 258280603d46496ebc5d7b3c8a34ed77, State: Initialized, Role: FOLLOWER
I20260619 14:59:42.000496 25563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } }
I20260619 14:59:42.001843 25687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } } }
I20260619 14:59:42.002573 25687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:59:42.003134 25613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [term 1 FOLLOWER]: Refusing update from remote peer ef5087f0e9284f0bb6c0a3378f530766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:59:42.006044 25682 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:59:42.009692 25563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:59:42.010097 25536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:42.009950 25537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef5087f0e9284f0bb6c0a3378f530766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "258280603d46496ebc5d7b3c8a34ed77"
I20260619 14:59:42.010738 25538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef5087f0e9284f0bb6c0a3378f530766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "258280603d46496ebc5d7b3c8a34ed77" is_pre_election: true
I20260619 14:59:42.014520 25536 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [term 1 FOLLOWER]: Refusing update from remote peer ef5087f0e9284f0bb6c0a3378f530766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:59:42.015820 25682 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:59:42.035409 25687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef5087f0e9284f0bb6c0a3378f530766. Latest consensus state: current_term: 1 leader_uuid: "ef5087f0e9284f0bb6c0a3378f530766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } } }
I20260619 14:59:42.035823 25687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:59:42.050189 25687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef5087f0e9284f0bb6c0a3378f530766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } } }
I20260619 14:59:42.050580 25687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:59:42.053347 25655 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef5087f0e9284f0bb6c0a3378f530766. Latest consensus state: current_term: 1 leader_uuid: "ef5087f0e9284f0bb6c0a3378f530766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } } }
I20260619 14:59:42.053781 25655 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:59:42.058784 25669 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef5087f0e9284f0bb6c0a3378f530766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } } }
I20260619 14:59:42.059154 25669 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:42.059445 25669 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef5087f0e9284f0bb6c0a3378f530766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } } }
I20260619 14:59:42.059728 25669 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:42.060860 25685 catalog_manager.cc:1347] Generated new cluster ID: 38264cf9b5264607ab677a44c60a5c2b
I20260619 14:59:42.060964 25685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:59:42.064074 25655 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef5087f0e9284f0bb6c0a3378f530766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef5087f0e9284f0bb6c0a3378f530766" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38871 } } peers { permanent_uuid: "258280603d46496ebc5d7b3c8a34ed77" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 43425 } } peers { permanent_uuid: "ea4967da854e47708edc1f4be95d6255" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 37249 } } }
I20260619 14:59:42.064325 25655 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 14:59:42.087080 25685 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:59:42.089049 25685 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:59:42.111606 25685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef5087f0e9284f0bb6c0a3378f530766: Generated new TSK 0
I20260619 14:59:42.112756 25685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 14:59:42.134976 25705 catalog_manager.cc:1259] Loaded cluster ID: 38264cf9b5264607ab677a44c60a5c2b
I20260619 14:59:42.135085 25705 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77: loading cluster ID for follower catalog manager: success
I20260619 14:59:42.138772 25705 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77: acquiring CA information for follower catalog manager: success
I20260619 14:59:42.141321 25705 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 258280603d46496ebc5d7b3c8a34ed77: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 14:59:42.843504 25666 catalog_manager.cc:1259] Loaded cluster ID: 38264cf9b5264607ab677a44c60a5c2b
I20260619 14:59:42.843606 25666 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255: loading cluster ID for follower catalog manager: success
I20260619 14:59:42.846187 25666 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255: acquiring CA information for follower catalog manager: success
I20260619 14:59:42.848145 25666 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ea4967da854e47708edc1f4be95d6255: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 14:59:43.255824 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 25422
I20260619 14:59:43.257157 25484 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:43.370086 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 25422
I20260619 14:59:43.391615 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 25496
I20260619 14:59:43.392838 25558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:43.501482 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 25496
I20260619 14:59:43.521997 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 25571
I20260619 14:59:43.523195 25633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:43.629982 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 25571
2026-06-19T14:59:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4803 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:59:43 dist-test-slave-76rk krb5kdc[25717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:59:43 dist-test-slave-76rk krb5kdc[25717](info): set up 2 sockets
Jun 19 14:59:43 dist-test-slave-76rk krb5kdc[25717](info): commencing operation
krb5kdc: starting...
W20260619 14:59:44.739339 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:59:44 dist-test-slave-76rk krb5kdc[25717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:59:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:59:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:59:44.998850 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38755
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:59:45.333869 25730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:45.334179 25730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:45.334273 25730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:45.341862 25730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:59:45.341981 25730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:45.342028 25730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:45.342082 25730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:59:45.342125 25730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:59:45.352140 25730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:45.353513 25730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:45.354772 25730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:45.366909 25736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:45.366928 25738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:45.367075 25735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:45.367882 25730 server_base.cc:1034] running on GCE node
I20260619 14:59:45.368788 25730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:45.370296 25730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:45.371569 25730 hybrid_clock.cc:648] HybridClock initialized: now 1781881185371508 us; error 57 us; skew 500 ppm
I20260619 14:59:45.371958 25730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:45 dist-test-slave-76rk krb5kdc[25717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881185, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:45.375293 25730 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:59:45.376883 25730 webserver.cc:460] Webserver started at http://127.11.175.254:36861/ using document root <none> and password file <none>
I20260619 14:59:45.377607 25730 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:45.377737 25730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:45.378070 25730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:59:45.381186 25730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "57c2d39c6771439f832d3c5d92837f59"
format_stamp: "Formatted at 2026-06-19 14:59:45 on dist-test-slave-76rk"
server_key: "241ed32a292a30dce120635519853ff1"
server_key_iv: "ce89831b944072698f64fd3162d9dd53"
server_key_version: "encryptionkey@0"
I20260619 14:59:45.381979 25730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "57c2d39c6771439f832d3c5d92837f59"
format_stamp: "Formatted at 2026-06-19 14:59:45 on dist-test-slave-76rk"
server_key: "241ed32a292a30dce120635519853ff1"
server_key_iv: "ce89831b944072698f64fd3162d9dd53"
server_key_version: "encryptionkey@0"
I20260619 14:59:45.387894 25730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260619 14:59:45.391690 25745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:45.393162 25730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:59:45.393355 25730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "57c2d39c6771439f832d3c5d92837f59"
format_stamp: "Formatted at 2026-06-19 14:59:45 on dist-test-slave-76rk"
server_key: "241ed32a292a30dce120635519853ff1"
server_key_iv: "ce89831b944072698f64fd3162d9dd53"
server_key_version: "encryptionkey@0"
I20260619 14:59:45.393567 25730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:45.424849 25730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:45.429582 25730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:45.429920 25730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:45.450299 25730 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:38755
I20260619 14:59:45.450347 25796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:38755 every 8 connection(s)
I20260619 14:59:45.451819 25730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:59:45.453524 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 25730
I20260619 14:59:45.453761 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:59:45.454617 11967 external_mini_cluster.cc:1428] Setting key 0e34f90003001af6cb0a497f33af15db
I20260619 14:59:45.456902 25797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 14:59:45 dist-test-slave-76rk krb5kdc[25717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:45.471035 25797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59: Bootstrap starting.
I20260619 14:59:45.475219 25797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:45.476380 25797 log.cc:824] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:45.479972 25797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59: No bootstrap required, opened a new log
I20260619 14:59:45.480218 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:45.456585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.11.175.254:38755)
0619 14:59:45.457397 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:45.457408 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:45.457448 (+ 40us) server_negotiation.cc:407] Connection header received
0619 14:59:45.459167 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:45.459189 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:45.459591 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:45.459929 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:45.462002 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:45.463494 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:45.465356 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:45.466096 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:45.466240 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:45.471585 (+ 5345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:45.471597 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:45.471613 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:45.471654 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:45.475465 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:45.476285 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:45.476294 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:45.476304 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:45.476514 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:45.477035 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:45.477043 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:45.477048 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:45.477907 (+ 859us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:45.478162 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:45.478722 (+ 560us) server_negotiation.cc:299] Negotiation successful
0619 14:59:45.479160 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":609,"thread_start_us":388,"threads_started":1}
I20260619 14:59:45.484382 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 14:59:45.488811 25797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c2d39c6771439f832d3c5d92837f59" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38755 } }
I20260619 14:59:45.489223 25797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:45.489337 25797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57c2d39c6771439f832d3c5d92837f59, State: Initialized, Role: FOLLOWER
I20260619 14:59:45.489974 25797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c2d39c6771439f832d3c5d92837f59" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38755 } }
I20260619 14:59:45.490224 25797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:59:45.490383 25797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:59:45.490559 25797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:45.492767 25797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c2d39c6771439f832d3c5d92837f59" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38755 } }
I20260619 14:59:45.493291 25797 leader_election.cc:302] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57c2d39c6771439f832d3c5d92837f59; no voters:
I20260619 14:59:45.493916 25797 leader_election.cc:288] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:59:45.494086 25802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:59:45.494830 25802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [term 1 LEADER]: Becoming Leader. State: Replica: 57c2d39c6771439f832d3c5d92837f59, State: Running, Role: LEADER
I20260619 14:59:45.495362 25802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c2d39c6771439f832d3c5d92837f59" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38755 } }
I20260619 14:59:45.496143 25797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:59:45.498281 25804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57c2d39c6771439f832d3c5d92837f59. Latest consensus state: current_term: 1 leader_uuid: "57c2d39c6771439f832d3c5d92837f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c2d39c6771439f832d3c5d92837f59" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38755 } } }
I20260619 14:59:45.498311 25803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57c2d39c6771439f832d3c5d92837f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57c2d39c6771439f832d3c5d92837f59" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38755 } } }
I20260619 14:59:45.498805 25804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:45.498950 25803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:45.500629 25807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:59:45.506023 25807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:59:45.515875 25807 catalog_manager.cc:1347] Generated new cluster ID: ba2428f7bbb448759bac0de2e0d97fb1
I20260619 14:59:45.516021 25807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:59:45.527875 25807 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:59:45.529428 25807 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:59:45.536813 25807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57c2d39c6771439f832d3c5d92837f59: Generated new TSK 0
I20260619 14:59:45.537791 25807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 14:59:45.598031 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 25730
I20260619 14:59:45.599448 25792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:45.697703 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 25730
2026-06-19T14:59:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2062 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 14:59:45 dist-test-slave-76rk krb5kdc[25825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 14:59:45 dist-test-slave-76rk krb5kdc[25825](info): set up 2 sockets
Jun 19 14:59:45 dist-test-slave-76rk krb5kdc[25825](info): commencing operation
krb5kdc: starting...
W20260619 14:59:46.819607 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.008s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 14:59:46 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T14:59:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T14:59:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 14:59:47.074024 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:42475
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:42475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:59:47.408712 25838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:47.409027 25838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:47.409201 25838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:47.416877 25838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:59:47.417011 25838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:47.417104 25838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:47.417186 25838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:59:47.417256 25838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:59:47.427670 25838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:42475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:42475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:47.429152 25838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:47.430601 25838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:47.443126 25846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:47.443148 25844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:47.443415 25843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:47.443959 25838 server_base.cc:1034] running on GCE node
I20260619 14:59:47.444796 25838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:47.446367 25838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:47.447613 25838 hybrid_clock.cc:648] HybridClock initialized: now 1781881187447533 us; error 75 us; skew 500 ppm
I20260619 14:59:47.447996 25838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:47 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:47.451233 25838 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:59:47.452833 25838 webserver.cc:460] Webserver started at http://127.11.175.254:37253/ using document root <none> and password file <none>
I20260619 14:59:47.453576 25838 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:47.453729 25838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:47.454056 25838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:59:47.457249 25838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "4762805edd364b98b49497e3e5371ea1"
format_stamp: "Formatted at 2026-06-19 14:59:47 on dist-test-slave-76rk"
server_key: "620d875c17c0c0822ffc64bc6029a7ab"
server_key_iv: "b697a3b1155e050b436ba2bf1f195dd4"
server_key_version: "encryptionkey@0"
I20260619 14:59:47.458079 25838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "4762805edd364b98b49497e3e5371ea1"
format_stamp: "Formatted at 2026-06-19 14:59:47 on dist-test-slave-76rk"
server_key: "620d875c17c0c0822ffc64bc6029a7ab"
server_key_iv: "b697a3b1155e050b436ba2bf1f195dd4"
server_key_version: "encryptionkey@0"
I20260619 14:59:47.463485 25838 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260619 14:59:47.467146 25853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:47.468649 25838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 14:59:47.468851 25838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "4762805edd364b98b49497e3e5371ea1"
format_stamp: "Formatted at 2026-06-19 14:59:47 on dist-test-slave-76rk"
server_key: "620d875c17c0c0822ffc64bc6029a7ab"
server_key_iv: "b697a3b1155e050b436ba2bf1f195dd4"
server_key_version: "encryptionkey@0"
I20260619 14:59:47.469081 25838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:47.492966 25838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:47.497721 25838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:47.498024 25838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:47.516783 25838 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:42475
I20260619 14:59:47.516836 25904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:42475 every 8 connection(s)
I20260619 14:59:47.518203 25838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:59:47.520128 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 25838
I20260619 14:59:47.520361 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 14:59:47.521078 11967 external_mini_cluster.cc:1428] Setting key 4827ad763deaeaa805d64e964a038d81
I20260619 14:59:47.522514 25905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 14:59:47 dist-test-slave-76rk krb5kdc[25825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:47.536834 25905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Bootstrap starting.
I20260619 14:59:47.541062 25905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:47.542464 25905 log.cc:824] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:47.545408 25905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: No bootstrap required, opened a new log
I20260619 14:59:47.546445 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:47.523362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.11.175.254:42475)
0619 14:59:47.524116 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:47.524132 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:47.524181 (+ 49us) server_negotiation.cc:407] Connection header received
0619 14:59:47.526164 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:47.526194 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:47.526761 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:47.527254 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:47.528657 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:47.530774 (+ 2117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:47.532207 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:47.533319 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:47.533531 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:47.537941 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:47.537952 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:47.537972 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:47.538031 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:47.542150 (+ 4119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:47.543033 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:47.543040 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:47.543047 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:47.543256 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:47.543679 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:47.543684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:47.543686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:47.544255 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:47.544567 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:47.544994 (+ 427us) server_negotiation.cc:299] Negotiation successful
0619 14:59:47.545421 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":308,"threads_started":1}
I20260619 14:59:47.553665 25905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:47.554090 25905 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:47.554198 25905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4762805edd364b98b49497e3e5371ea1, State: Initialized, Role: FOLLOWER
I20260619 14:59:47.554883 25905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:47.555171 25905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:59:47.555335 25905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:59:47.555547 25905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:47.557889 25905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:47.558396 25905 leader_election.cc:302] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4762805edd364b98b49497e3e5371ea1; no voters:
I20260619 14:59:47.559005 25905 leader_election.cc:288] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 14:59:47.559159 25910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:59:47.560418 25910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 1 LEADER]: Becoming Leader. State: Replica: 4762805edd364b98b49497e3e5371ea1, State: Running, Role: LEADER
I20260619 14:59:47.560976 25910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:47.561182 25905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:59:47.564073 25911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4762805edd364b98b49497e3e5371ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } } }
I20260619 14:59:47.564463 25911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:47.564159 25912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4762805edd364b98b49497e3e5371ea1. Latest consensus state: current_term: 1 leader_uuid: "4762805edd364b98b49497e3e5371ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } } }
I20260619 14:59:47.564591 25912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:47.566315 25917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:59:47.571985 25917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:59:47.581354 25917 catalog_manager.cc:1347] Generated new cluster ID: 92030b6128e44d118743f02960b33260
I20260619 14:59:47.581481 25917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:59:47.615279 25917 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 14:59:47.616833 25917 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:59:47.628754 25917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Generated new TSK 0
I20260619 14:59:47.629912 25917 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 14:59:47.695721 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42475
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:59:48.036252 25931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:48.036626 25931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:48.036856 25931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:48.044629 25931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:48.044790 25931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:48.044981 25931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:59:48.054946 25931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:48.056214 25931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:48.057526 25931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:48.071347 25936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:48.071296 25939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:48.071300 25937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:48.072520 25931 server_base.cc:1034] running on GCE node
I20260619 14:59:48.073457 25931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:48.074507 25931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:48.075747 25931 hybrid_clock.cc:648] HybridClock initialized: now 1781881188075680 us; error 61 us; skew 500 ppm
I20260619 14:59:48.076133 25931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:48 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881188, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:48.079965 25931 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:59:48.082369 25931 webserver.cc:460] Webserver started at http://127.11.175.193:33231/ using document root <none> and password file <none>
I20260619 14:59:48.083267 25931 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:48.083436 25931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:48.083771 25931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:59:48.087014 25931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "259102544c184ea6a3aa6252e81f3ca4"
format_stamp: "Formatted at 2026-06-19 14:59:48 on dist-test-slave-76rk"
server_key: "47a0cf780b6526361d43c75f0051382c"
server_key_iv: "9719d972aa2828867d7ac8863a00aa2a"
server_key_version: "encryptionkey@0"
I20260619 14:59:48.087872 25931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "259102544c184ea6a3aa6252e81f3ca4"
format_stamp: "Formatted at 2026-06-19 14:59:48 on dist-test-slave-76rk"
server_key: "47a0cf780b6526361d43c75f0051382c"
server_key_iv: "9719d972aa2828867d7ac8863a00aa2a"
server_key_version: "encryptionkey@0"
I20260619 14:59:48.093526 25931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 14:59:48.097313 25946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:48.098866 25931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:59:48.099053 25931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "259102544c184ea6a3aa6252e81f3ca4"
format_stamp: "Formatted at 2026-06-19 14:59:48 on dist-test-slave-76rk"
server_key: "47a0cf780b6526361d43c75f0051382c"
server_key_iv: "9719d972aa2828867d7ac8863a00aa2a"
server_key_version: "encryptionkey@0"
I20260619 14:59:48.099268 25931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:48.163378 25931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:48.168143 25931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:48.168485 25931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:48.169567 25931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:48.171113 25931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:48.171226 25931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:48.171399 25931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:48.171501 25931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:48.208947 25931 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:36347
I20260619 14:59:48.209000 26058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:36347 every 8 connection(s)
I20260619 14:59:48.210350 25931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:59:48.212605 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 25931
I20260619 14:59:48.212852 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 14:59:48.213763 11967 external_mini_cluster.cc:1428] Setting key 6d8ae552214f0c1c3769ed752a7b1206
Jun 19 14:59:48 dist-test-slave-76rk krb5kdc[25825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881188, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:48.234774 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:48.213364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:51163 (local address 127.11.175.254:42475)
0619 14:59:48.213865 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:48.213871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:48.215380 (+ 1509us) server_negotiation.cc:407] Connection header received
0619 14:59:48.217576 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:48.217584 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:48.217700 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:48.217828 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:48.219574 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:48.220797 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:48.222527 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:48.223250 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:48.223396 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:48.228005 (+ 4609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:48.228012 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:48.228015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:48.228052 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:48.231112 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:48.232162 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:48.232166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:48.232168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:48.232291 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:48.232822 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:48.232825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:48.232827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:48.233093 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:48.233218 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:48.234245 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0619 14:59:48.234459 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":264,"threads_started":1}
I20260619 14:59:48.236356 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:48.214183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:51163)
0619 14:59:48.215178 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:48.215235 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:48.217267 (+ 2032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:48.218082 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:48.218091 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:48.218763 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:48.219389 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:48.219409 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:48.220968 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:48.220972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:48.222352 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:48.222366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:48.223468 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:48.223473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:48.223686 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:48.224476 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:48.224500 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:48.227738 (+ 3238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:48.231295 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:48.231308 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:48.231328 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:48.231999 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:48.232420 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:48.232428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:48.232435 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:48.232701 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:48.233237 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:48.233249 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:48.233770 (+ 521us) client_negotiation.cc:769] Sending connection context
0619 14:59:48.234094 (+ 324us) client_negotiation.cc:240] Negotiation successful
0619 14:59:48.234755 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":458,"threads_started":1}
I20260619 14:59:48.238202 26059 heartbeater.cc:347] Connected to a master server at 127.11.175.254:42475
I20260619 14:59:48.238579 26059 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:48.239367 26059 heartbeater.cc:511] Master 127.11.175.254:42475 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 14:59:48.241613 25870 ts_manager.cc:194] Registered new tserver with Master: 259102544c184ea6a3aa6252e81f3ca4 (127.11.175.193:36347)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
I20260619 14:59:48.243427 25870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:51163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 14:59:48.272462 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42475
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:59:48.600189 26065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:48.600536 26065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:48.600719 26065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:48.608466 26065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:48.608629 26065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:48.608803 26065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:59:48.618775 26065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:48.620092 26065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:48.621431 26065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:48.635192 26071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:48.635397 26070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:48.635401 26073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:48.636425 26065 server_base.cc:1034] running on GCE node
I20260619 14:59:48.636997 26065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:48.637952 26065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:48.639149 26065 hybrid_clock.cc:648] HybridClock initialized: now 1781881188639097 us; error 53 us; skew 500 ppm
I20260619 14:59:48.639503 26065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:48 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881188, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:48.642897 26065 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:59:48.644555 26065 webserver.cc:460] Webserver started at http://127.11.175.194:44719/ using document root <none> and password file <none>
I20260619 14:59:48.645236 26065 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:48.645349 26065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:48.645690 26065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:59:48.648802 26065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "70f0f027df57489b8a09cad395699015"
format_stamp: "Formatted at 2026-06-19 14:59:48 on dist-test-slave-76rk"
server_key: "7a62622c53759265872ca4f37571c639"
server_key_iv: "fc168f780f24ff42a24fff57dfe4f472"
server_key_version: "encryptionkey@0"
I20260619 14:59:48.649576 26065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "70f0f027df57489b8a09cad395699015"
format_stamp: "Formatted at 2026-06-19 14:59:48 on dist-test-slave-76rk"
server_key: "7a62622c53759265872ca4f37571c639"
server_key_iv: "fc168f780f24ff42a24fff57dfe4f472"
server_key_version: "encryptionkey@0"
I20260619 14:59:48.655061 26065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 14:59:48.658921 26080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:48.660316 26065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 14:59:48.660501 26065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "70f0f027df57489b8a09cad395699015"
format_stamp: "Formatted at 2026-06-19 14:59:48 on dist-test-slave-76rk"
server_key: "7a62622c53759265872ca4f37571c639"
server_key_iv: "fc168f780f24ff42a24fff57dfe4f472"
server_key_version: "encryptionkey@0"
I20260619 14:59:48.660693 26065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:48.678351 26065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:48.682955 26065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:48.683255 26065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:48.684255 26065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:48.685642 26065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:48.685715 26065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:48.685834 26065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:48.685904 26065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:48.719614 26065 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:41057
I20260619 14:59:48.719656 26192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:41057 every 8 connection(s)
I20260619 14:59:48.720909 26065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:59:48.727411 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 26065
I20260619 14:59:48.727622 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 14:59:48.728171 11967 external_mini_cluster.cc:1428] Setting key 50484806795fb84fad068ed95f5bec13
Jun 19 14:59:48 dist-test-slave-76rk krb5kdc[25825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881188, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:48.744887 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:48.723518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:53821 (local address 127.11.175.254:42475)
0619 14:59:48.723750 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:48.723757 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:48.724962 (+ 1205us) server_negotiation.cc:407] Connection header received
0619 14:59:48.726887 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:48.726892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:48.726966 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:48.727057 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:48.729189 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:48.730262 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:48.733002 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:48.733434 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:48.733536 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:48.738346 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:48.738356 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:48.738363 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:48.738418 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:48.741400 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:48.742173 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:48.742181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:48.742188 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:48.742364 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:48.742867 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:48.742875 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:48.742881 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:48.743291 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:48.743457 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:48.744145 (+ 688us) server_negotiation.cc:299] Negotiation successful
0619 14:59:48.744437 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 14:59:48.745637 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:48.724018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:53821)
0619 14:59:48.724783 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:48.724827 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:48.726540 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:48.727375 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:48.727389 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:48.728204 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:48.728997 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:48.729026 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:48.730474 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:48.730480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:48.732824 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:48.732850 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:48.733677 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:48.733694 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:48.734035 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:48.734907 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:48.734950 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:48.738067 (+ 3117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:48.741543 (+ 3476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:48.741550 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:48.741564 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:48.742010 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:48.742476 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:48.742479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:48.742481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:48.742726 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:48.743413 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:48.743419 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:48.743752 (+ 333us) client_negotiation.cc:769] Sending connection context
0619 14:59:48.743988 (+ 236us) client_negotiation.cc:240] Negotiation successful
0619 14:59:48.744379 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":313,"threads_started":1}
I20260619 14:59:48.747213 26193 heartbeater.cc:347] Connected to a master server at 127.11.175.254:42475
I20260619 14:59:48.747552 26193 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:48.748471 26193 heartbeater.cc:511] Master 127.11.175.254:42475 requested a full tablet report, sending...
I20260619 14:59:48.750109 25870 ts_manager.cc:194] Registered new tserver with Master: 70f0f027df57489b8a09cad395699015 (127.11.175.194:41057)
I20260619 14:59:48.751020 25870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:53821
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 14:59:48.789685 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42475
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:59:49.126559 26198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:49.126907 26198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:49.127084 26198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:49.134292 26198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:49.134460 26198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:49.134670 26198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:59:49.144507 26198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:49.145991 26198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:49.147388 26198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:49.161194 26203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:49.161118 26204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:49.161084 26206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:49.162201 26198 server_base.cc:1034] running on GCE node
I20260619 14:59:49.162990 26198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:49.164019 26198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:49.165242 26198 hybrid_clock.cc:648] HybridClock initialized: now 1781881189165193 us; error 41 us; skew 500 ppm
I20260619 14:59:49.165614 26198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:49 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881189, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:49.168987 26198 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:59:49.170457 26198 webserver.cc:460] Webserver started at http://127.11.175.195:45725/ using document root <none> and password file <none>
I20260619 14:59:49.171197 26198 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:49.171346 26198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:49.171656 26198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 14:59:49.174765 26198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "fab9a33b44db47808f10e7e8407d63c3"
format_stamp: "Formatted at 2026-06-19 14:59:49 on dist-test-slave-76rk"
server_key: "2468343ff45c9f7af3d5403cfc2e7fb2"
server_key_iv: "daff209918535d4bb8197c0b9263b140"
server_key_version: "encryptionkey@0"
I20260619 14:59:49.175555 26198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "fab9a33b44db47808f10e7e8407d63c3"
format_stamp: "Formatted at 2026-06-19 14:59:49 on dist-test-slave-76rk"
server_key: "2468343ff45c9f7af3d5403cfc2e7fb2"
server_key_iv: "daff209918535d4bb8197c0b9263b140"
server_key_version: "encryptionkey@0"
I20260619 14:59:49.180869 26198 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 14:59:49.184480 26213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:49.185760 26198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 14:59:49.185932 26198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "fab9a33b44db47808f10e7e8407d63c3"
format_stamp: "Formatted at 2026-06-19 14:59:49 on dist-test-slave-76rk"
server_key: "2468343ff45c9f7af3d5403cfc2e7fb2"
server_key_iv: "daff209918535d4bb8197c0b9263b140"
server_key_version: "encryptionkey@0"
I20260619 14:59:49.186129 26198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:49.220405 26198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:49.224992 26198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:49.225292 26198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:49.226325 26198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:49.227746 26198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:49.227850 26198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:49.228000 26198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:49.228081 26198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:49.246588 26059 heartbeater.cc:503] Master 127.11.175.254:42475 was elected leader, sending a full tablet report...
I20260619 14:59:49.261714 26198 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:41247
I20260619 14:59:49.261793 26325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:41247 every 8 connection(s)
I20260619 14:59:49.263124 26198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:59:49.264964 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 26198
I20260619 14:59:49.265166 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 14:59:49.265657 11967 external_mini_cluster.cc:1428] Setting key 0e421e15de76b550d9ff6a16d6045598
Jun 19 14:59:49 dist-test-slave-76rk krb5kdc[25825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881189, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:49.285352 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.266086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:39417 (local address 127.11.175.254:42475)
0619 14:59:49.266806 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.266812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.269778 (+ 2966us) server_negotiation.cc:407] Connection header received
0619 14:59:49.269829 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.269832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.269902 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.269986 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:49.271922 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.272777 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.274545 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.275034 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.275138 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.278830 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:49.278837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:49.278840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:49.278879 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:49.281779 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:49.282585 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:49.282589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:49.282591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:49.282838 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:49.283321 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:49.283326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:49.283328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:49.283659 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:49.283858 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.284827 (+ 969us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.285046 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":444,"threads_started":1}
I20260619 14:59:49.286324 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.266816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:39417)
0619 14:59:49.267769 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:49.267822 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:49.269496 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:49.270295 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:49.270305 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:49.271004 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:49.271630 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.271650 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.272936 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.272941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.274379 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.274392 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.275244 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.275250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.275462 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.276143 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:49.276165 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:49.278454 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:49.281922 (+ 3468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:49.281932 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:49.281949 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:49.282468 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:49.282946 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:49.282950 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:49.282952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:49.283171 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:49.283920 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:49.283927 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:49.284368 (+ 441us) client_negotiation.cc:769] Sending connection context
0619 14:59:49.284641 (+ 273us) client_negotiation.cc:240] Negotiation successful
0619 14:59:49.285076 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":473,"threads_started":1}
I20260619 14:59:49.287906 26326 heartbeater.cc:347] Connected to a master server at 127.11.175.254:42475
I20260619 14:59:49.288275 26326 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:49.289079 26326 heartbeater.cc:511] Master 127.11.175.254:42475 requested a full tablet report, sending...
I20260619 14:59:49.290948 25870 ts_manager.cc:194] Registered new tserver with Master: fab9a33b44db47808f10e7e8407d63c3 (127.11.175.195:41247)
I20260619 14:59:49.291774 25870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:39417
I20260619 14:59:49.294402 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 14:59:49.312565 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.299147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.11.175.254:42475)
0619 14:59:49.299332 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.299337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.299760 (+ 423us) server_negotiation.cc:407] Connection header received
0619 14:59:49.299925 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.299929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.299991 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.300077 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:49.301475 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.302221 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.303733 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.304256 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.304365 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.306861 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:49.306867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:49.306870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:49.306905 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:49.309598 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:49.310327 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:49.310333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:49.310337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:49.310499 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:49.311023 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:49.311027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:49.311029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:49.311384 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:49.311573 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.312137 (+ 564us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.312316 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 14:59:49.317209 25870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:59:49.324267 25870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:59:49.347777 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.337284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:41057 (local address 127.0.0.1:57384)
0619 14:59:49.338751 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:49.338768 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:49.338876 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:49.340763 (+ 1887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:49.340774 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:49.340818 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:49.341253 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.341268 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.343205 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.343212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.344763 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.344778 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.346741 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.346747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.347096 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.347174 (+ 78us) client_negotiation.cc:769] Sending connection context
0619 14:59:49.347305 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 14:59:49.347464 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"mutex_wait_us":67,"thread_start_us":4755,"threads_started":1}
I20260619 14:59:49.348663 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.338324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57384 (local address 127.11.175.194:41057)
0619 14:59:49.339944 (+ 1620us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.339956 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.339994 (+ 38us) server_negotiation.cc:407] Connection header received
0619 14:59:49.340128 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.340140 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.340471 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.340721 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:49.341462 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.343033 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.344975 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.346514 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.346694 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.346875 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.348295 (+ 1420us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.348433 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1460,"thread_start_us":1358,"threads_started":1}
I20260619 14:59:49.349920 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.337159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.0.0.1:43716)
0619 14:59:49.338101 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:49.338147 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:49.338332 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:49.342077 (+ 3745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:49.342082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:49.342111 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:49.342537 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.342552 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.344790 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.344795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.347218 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.347230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.349069 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.349076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.349247 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.349284 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 14:59:49.349362 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 14:59:49.349493 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":84,"thread_start_us":476,"threads_started":1}
I20260619 14:59:49.350548 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.338869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.11.175.195:41247)
0619 14:59:49.339764 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.339776 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.342021 (+ 2245us) server_negotiation.cc:407] Connection header received
0619 14:59:49.342116 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.342123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.342341 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.342545 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:49.343141 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.344704 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.347349 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.348822 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.349032 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.349218 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.350027 (+ 809us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.350204 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":459,"threads_started":1}
I20260619 14:59:49.351066 26340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.337975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:41247 (local address 127.0.0.1:43740)
0619 14:59:49.341747 (+ 3772us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:49.341764 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:49.341888 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:49.342554 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:49.342560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:49.342584 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:49.342949 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.342962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.344869 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.344874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.347158 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.347176 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.349383 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.349389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.349535 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.349564 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 14:59:49.349645 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 14:59:49.349766 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3666,"mutex_wait_us":153,"thread_start_us":455,"threads_started":1}
I20260619 14:59:49.351114 26341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.337916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.11.175.193:36347)
0619 14:59:49.341471 (+ 3555us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.341483 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.341515 (+ 32us) server_negotiation.cc:407] Connection header received
0619 14:59:49.341613 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.341620 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.341874 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.342783 (+ 909us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:49.342984 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.344614 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.347429 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.348853 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.350317 (+ 1464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.350500 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.350571 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.350771 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3370,"thread_start_us":3227,"threads_started":1}
I20260619 14:59:49.352758 26128 tablet_service.cc:1467] Processing CreateTablet for tablet e7d347a9fde74612aa80d88ce79c7a04 (DEFAULT_TABLE table=test-table [id=772c85862b0340cc88da4d3b535dd218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:59:49.354097 26128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d347a9fde74612aa80d88ce79c7a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:49.354583 26261 tablet_service.cc:1467] Processing CreateTablet for tablet e7d347a9fde74612aa80d88ce79c7a04 (DEFAULT_TABLE table=test-table [id=772c85862b0340cc88da4d3b535dd218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:59:49.354584 25994 tablet_service.cc:1467] Processing CreateTablet for tablet e7d347a9fde74612aa80d88ce79c7a04 (DEFAULT_TABLE table=test-table [id=772c85862b0340cc88da4d3b535dd218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:59:49.356060 25994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d347a9fde74612aa80d88ce79c7a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:49.356060 26261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d347a9fde74612aa80d88ce79c7a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:49.366493 26345 tablet_bootstrap.cc:492] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: Bootstrap starting.
I20260619 14:59:49.370258 26345 tablet_bootstrap.cc:654] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:49.371052 26346 tablet_bootstrap.cc:492] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: Bootstrap starting.
I20260619 14:59:49.371783 26345 log.cc:824] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:49.371955 26347 tablet_bootstrap.cc:492] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: Bootstrap starting.
I20260619 14:59:49.375147 26346 tablet_bootstrap.cc:654] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:49.375993 26347 tablet_bootstrap.cc:654] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:49.376310 26346 log.cc:824] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:49.377569 26345 tablet_bootstrap.cc:492] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: No bootstrap required, opened a new log
I20260619 14:59:49.377521 26347 log.cc:824] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:49.377866 26345 ts_tablet_manager.cc:1397] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260619 14:59:49.379751 26346 tablet_bootstrap.cc:492] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: No bootstrap required, opened a new log
I20260619 14:59:49.380024 26346 ts_tablet_manager.cc:1397] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260619 14:59:49.381208 26347 tablet_bootstrap.cc:492] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: No bootstrap required, opened a new log
I20260619 14:59:49.381472 26347 ts_tablet_manager.cc:1397] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260619 14:59:49.386926 26346 raft_consensus.cc:348] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } }
I20260619 14:59:49.387327 26346 raft_consensus.cc:374] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:49.387475 26346 raft_consensus.cc:729] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab9a33b44db47808f10e7e8407d63c3, State: Initialized, Role: FOLLOWER
I20260619 14:59:49.388399 26346 consensus_queue.cc:260] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } }
I20260619 14:59:49.388621 26345 raft_consensus.cc:348] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } }
I20260619 14:59:49.389128 26345 raft_consensus.cc:374] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:49.389262 26345 raft_consensus.cc:729] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70f0f027df57489b8a09cad395699015, State: Initialized, Role: FOLLOWER
I20260619 14:59:49.389976 26345 consensus_queue.cc:260] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } }
I20260619 14:59:49.390206 26326 heartbeater.cc:503] Master 127.11.175.254:42475 was elected leader, sending a full tablet report...
I20260619 14:59:49.391258 26193 heartbeater.cc:503] Master 127.11.175.254:42475 was elected leader, sending a full tablet report...
I20260619 14:59:49.391933 26345 ts_tablet_manager.cc:1428] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
I20260619 14:59:49.392185 26347 raft_consensus.cc:348] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } }
I20260619 14:59:49.392675 26347 raft_consensus.cc:374] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:49.392809 26347 raft_consensus.cc:729] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 259102544c184ea6a3aa6252e81f3ca4, State: Initialized, Role: FOLLOWER
I20260619 14:59:49.393421 26347 consensus_queue.cc:260] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } }
I20260619 14:59:49.395334 26347 ts_tablet_manager.cc:1428] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260619 14:59:49.395834 26346 ts_tablet_manager.cc:1428] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: Time spent starting tablet: real 0.016s user 0.007s sys 0.004s
I20260619 14:59:49.417510 26351 raft_consensus.cc:482] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:59:49.417874 26351 raft_consensus.cc:504] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } }
I20260619 14:59:49.419859 26351 leader_election.cc:288] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 259102544c184ea6a3aa6252e81f3ca4 (127.11.175.193:36347), 70f0f027df57489b8a09cad395699015 (127.11.175.194:41057)
I20260619 14:59:49.426141 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.420055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.11.175.195:56825)
0619 14:59:49.420344 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:49.420385 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:49.420537 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:49.420941 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:49.420945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:49.420966 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:49.421190 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.421201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.422445 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.422450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.423976 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.423988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.425315 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.425321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.425513 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.425549 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 14:59:49.425653 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 14:59:49.425793 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260619 14:59:49.426208 26341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.420158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:56825 (local address 127.11.175.193:36347)
0619 14:59:49.420344 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.420350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.420576 (+ 226us) server_negotiation.cc:407] Connection header received
0619 14:59:49.420704 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.420712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.420800 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.420904 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:49.421338 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.422305 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.424139 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.425168 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.425290 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.425342 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.425774 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.425921 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 14:59:49.427026 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.420644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:39991 (local address 127.11.175.194:41057)
0619 14:59:49.420839 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.420843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.421039 (+ 196us) server_negotiation.cc:407] Connection header received
0619 14:59:49.421216 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.421220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.421320 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.421404 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:49.421974 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.422870 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.424811 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.425813 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.425956 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.426035 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.426574 (+ 539us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.426758 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 14:59:49.427060 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.420132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:41057 (local address 127.11.175.195:39991)
0619 14:59:49.420940 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:49.420959 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:49.421088 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:49.421454 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:49.421458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:49.421485 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:49.421830 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.421842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.423028 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.423032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.424642 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:49.424656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.426045 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.426052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:49.426285 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.426355 (+ 70us) client_negotiation.cc:769] Sending connection context
0619 14:59:49.426544 (+ 189us) client_negotiation.cc:240] Negotiation successful
0619 14:59:49.426700 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":316,"threads_started":1}
I20260619 14:59:49.427352 26014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7d347a9fde74612aa80d88ce79c7a04" candidate_uuid: "fab9a33b44db47808f10e7e8407d63c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "259102544c184ea6a3aa6252e81f3ca4" is_pre_election: true
I20260619 14:59:49.427968 26014 raft_consensus.cc:2454] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fab9a33b44db47808f10e7e8407d63c3 in term 0.
I20260619 14:59:49.428083 26148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7d347a9fde74612aa80d88ce79c7a04" candidate_uuid: "fab9a33b44db47808f10e7e8407d63c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70f0f027df57489b8a09cad395699015" is_pre_election: true
I20260619 14:59:49.428565 26148 raft_consensus.cc:2454] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fab9a33b44db47808f10e7e8407d63c3 in term 0.
I20260619 14:59:49.428726 26217 leader_election.cc:302] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 259102544c184ea6a3aa6252e81f3ca4, fab9a33b44db47808f10e7e8407d63c3; no voters:
I20260619 14:59:49.429158 26351 raft_consensus.cc:2781] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:59:49.429275 26351 raft_consensus.cc:482] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:59:49.429397 26351 raft_consensus.cc:3037] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:49.431648 26351 raft_consensus.cc:504] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } }
I20260619 14:59:49.432279 26351 leader_election.cc:288] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [CANDIDATE]: Term 1 election: Requested vote from peers 259102544c184ea6a3aa6252e81f3ca4 (127.11.175.193:36347), 70f0f027df57489b8a09cad395699015 (127.11.175.194:41057)
I20260619 14:59:49.432741 26014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7d347a9fde74612aa80d88ce79c7a04" candidate_uuid: "fab9a33b44db47808f10e7e8407d63c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "259102544c184ea6a3aa6252e81f3ca4"
I20260619 14:59:49.432739 26148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7d347a9fde74612aa80d88ce79c7a04" candidate_uuid: "fab9a33b44db47808f10e7e8407d63c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70f0f027df57489b8a09cad395699015"
I20260619 14:59:49.432946 26014 raft_consensus.cc:3037] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:49.432946 26148 raft_consensus.cc:3037] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:49.436146 26148 raft_consensus.cc:2454] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fab9a33b44db47808f10e7e8407d63c3 in term 1.
I20260619 14:59:49.436146 26014 raft_consensus.cc:2454] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fab9a33b44db47808f10e7e8407d63c3 in term 1.
I20260619 14:59:49.436585 26214 leader_election.cc:302] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70f0f027df57489b8a09cad395699015, fab9a33b44db47808f10e7e8407d63c3; no voters:
I20260619 14:59:49.436925 26351 raft_consensus.cc:2781] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:59:49.437486 26351 raft_consensus.cc:686] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 1 LEADER]: Becoming Leader. State: Replica: fab9a33b44db47808f10e7e8407d63c3, State: Running, Role: LEADER
I20260619 14:59:49.438012 26351 consensus_queue.cc:237] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } }
I20260619 14:59:49.442451 25869 catalog_manager.cc:5495] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to fab9a33b44db47808f10e7e8407d63c3 (127.11.175.195). New cstate: current_term: 1 leader_uuid: "fab9a33b44db47808f10e7e8407d63c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } health_report { overall_health: HEALTHY } } }
W20260619 14:59:49.462985 26060 tablet.cc:2369] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 14:59:49.472886 26194 tablet.cc:2369] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:59:49.475656 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.469993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43744 (local address 127.11.175.195:41247)
0619 14:59:49.470229 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.470236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.470256 (+ 20us) server_negotiation.cc:407] Connection header received
0619 14:59:49.470485 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.470493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.470575 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.470774 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:59:49.471178 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.471914 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.473426 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.473851 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.473992 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.474515 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:59:49.475080 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:59:49.475256 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.475314 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.475397 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 14:59:49.484063 26148 raft_consensus.cc:1261] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015 [term 1 FOLLOWER]: Refusing update from remote peer fab9a33b44db47808f10e7e8407d63c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:59:49.484063 26014 raft_consensus.cc:1261] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4 [term 1 FOLLOWER]: Refusing update from remote peer fab9a33b44db47808f10e7e8407d63c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:59:49.485167 26355 consensus_queue.cc:1035] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:59:49.485527 26351 consensus_queue.cc:1035] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:59:49.499516 26360 mvcc.cc:205] Tried to move back new op lower bound from 7298585352116318208 to 7298585351942746112. Current Snapshot: MvccSnapshot[applied={T|T < 7298585352116318208}]
I20260619 14:59:49.510588 25869 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:59:49.510951 25869 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:59:49.515224 25869 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Sending DeleteTablet for 3 replicas of tablet e7d347a9fde74612aa80d88ce79c7a04
I20260619 14:59:49.516568 25994 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d347a9fde74612aa80d88ce79c7a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:59:49 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:43716
I20260619 14:59:49.516644 26128 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d347a9fde74612aa80d88ce79c7a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:59:49 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:57384
I20260619 14:59:49.517657 26368 tablet_replica.cc:330] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: stopping tablet replica
I20260619 14:59:49.517864 26367 tablet_replica.cc:330] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: stopping tablet replica
I20260619 14:59:49.518158 26368 raft_consensus.cc:2229] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:59:49.518370 26367 raft_consensus.cc:2229] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:59:49.518797 26368 raft_consensus.cc:2258] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:59:49.519008 26367 raft_consensus.cc:2258] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:59:49.519084 26261 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d347a9fde74612aa80d88ce79c7a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:59:49 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:43740
I20260619 14:59:49.520150 26369 tablet_replica.cc:330] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: stopping tablet replica
I20260619 14:59:49.520633 26369 raft_consensus.cc:2229] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:59:49.520787 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 25931
I20260619 14:59:49.521494 26369 raft_consensus.cc:2258] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:59:49.521548 26368 ts_tablet_manager.cc:1905] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:59:49.529055 26368 ts_tablet_manager.cc:1918] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:59:49.529314 26368 log.cc:1196] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/e7d347a9fde74612aa80d88ce79c7a04
I20260619 14:59:49.529934 26368 ts_tablet_manager.cc:1939] T e7d347a9fde74612aa80d88ce79c7a04 P 259102544c184ea6a3aa6252e81f3ca4: Deleting consensus metadata
I20260619 14:59:49.531890 25857 catalog_manager.cc:4841] TS 259102544c184ea6a3aa6252e81f3ca4 (127.11.175.193:36347): tablet e7d347a9fde74612aa80d88ce79c7a04 (table test-table [id=772c85862b0340cc88da4d3b535dd218]) successfully deleted
I20260619 14:59:49.534780 26367 ts_tablet_manager.cc:1905] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 14:59:49 dist-test-slave-76rk krb5kdc[25825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 14:59:49.535310 26369 ts_tablet_manager.cc:1905] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:59:49.545215 26341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.522496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43718 (local address 127.11.175.193:36347)
0619 14:59:49.522765 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.522772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.522792 (+ 20us) server_negotiation.cc:407] Connection header received
0619 14:59:49.522861 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.522866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.522952 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.523066 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:49.525277 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.526625 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.528931 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.529670 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.529798 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.536742 (+ 6944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:49.536760 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:49.536781 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:49.536833 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:49.541120 (+ 4287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:49.542193 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:49.542203 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:49.542207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:49.542385 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:49.542961 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:49.542968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:49.542972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:49.543537 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:49.543838 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.544539 (+ 701us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.544796 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 14:59:49.546533 26367 ts_tablet_manager.cc:1918] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:59:49.547415 26054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:49.549484 26367 log.cc:1196] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/e7d347a9fde74612aa80d88ce79c7a04
I20260619 14:59:49.550084 26367 ts_tablet_manager.cc:1939] T e7d347a9fde74612aa80d88ce79c7a04 P 70f0f027df57489b8a09cad395699015: Deleting consensus metadata
I20260619 14:59:49.552126 25854 catalog_manager.cc:4841] TS 70f0f027df57489b8a09cad395699015 (127.11.175.194:41057): tablet e7d347a9fde74612aa80d88ce79c7a04 (table test-table [id=772c85862b0340cc88da4d3b535dd218]) successfully deleted
I20260619 14:59:49.553490 26369 ts_tablet_manager.cc:1918] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:59:49.553937 26369 log.cc:1196] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/e7d347a9fde74612aa80d88ce79c7a04
I20260619 14:59:49.555474 26369 ts_tablet_manager.cc:1939] T e7d347a9fde74612aa80d88ce79c7a04 P fab9a33b44db47808f10e7e8407d63c3: Deleting consensus metadata
I20260619 14:59:49.557904 25856 catalog_manager.cc:4841] TS fab9a33b44db47808f10e7e8407d63c3 (127.11.175.195:41247): tablet e7d347a9fde74612aa80d88ce79c7a04 (table test-table [id=772c85862b0340cc88da4d3b535dd218]) successfully deleted
I20260619 14:59:49.664870 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 25931
I20260619 14:59:49.682576 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 26065
Jun 19 14:59:49 dist-test-slave-76rk krb5kdc[25825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 14:59:49.702349 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.683470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57394 (local address 127.11.175.194:41057)
0619 14:59:49.683635 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.683640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.683658 (+ 18us) server_negotiation.cc:407] Connection header received
0619 14:59:49.683924 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.683932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.684040 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.684127 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:49.685933 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.686933 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.688859 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.689257 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.689362 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.695417 (+ 6055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:49.695439 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:49.695460 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:49.695511 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:49.699390 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:49.700141 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:49.700151 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:49.700155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:49.700321 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:49.700667 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:49.700672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:49.700676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:49.701174 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:49.701476 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.701748 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.701976 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 14:59:49.703147 26188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:49.792974 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 26065
I20260619 14:59:49.816449 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 26198
Jun 19 14:59:49 dist-test-slave-76rk krb5kdc[25825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 14:59:49.837805 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:49.817948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.11.175.195:41247)
0619 14:59:49.818238 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:49.818246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:49.818269 (+ 23us) server_negotiation.cc:407] Connection header received
0619 14:59:49.818408 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:49.818417 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:49.818568 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:49.818725 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:49.820593 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.822557 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.824768 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:49.825491 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:49.825641 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:49.830269 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:49.830287 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:49.830309 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:49.830374 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:49.834541 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:49.835378 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:49.835394 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:49.835399 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:49.835614 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:49.836015 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:49.836020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:49.836024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:49.836528 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:49.836842 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:49.837103 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 14:59:49.837355 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 14:59:49.838667 26321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:49.975389 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 26198
I20260619 14:59:49.997712 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 25838
I20260619 14:59:49.998931 25900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:50.113107 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 25838
I20260619 14:59:50.135421 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:42475
--webserver_interface=127.11.175.254
--webserver_port=37253
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:42475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:59:50.477027 26375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:50.477324 26375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:50.477506 26375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:50.485018 26375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:59:50.485133 26375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:50.485224 26375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:50.485311 26375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:59:50.485378 26375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:59:50.495492 26375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:42475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:42475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=37253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:50.496878 26375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:50.498328 26375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:50.523808 26384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:50.523808 26382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:50.524056 26381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:50.524345 26375 server_base.cc:1034] running on GCE node
I20260619 14:59:50.525497 26375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:50.527253 26375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:50.528512 26375 hybrid_clock.cc:648] HybridClock initialized: now 1781881190528459 us; error 63 us; skew 500 ppm
I20260619 14:59:50.528986 26375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:50 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:50.532804 26375 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:59:50.534484 26375 webserver.cc:460] Webserver started at http://127.11.175.254:37253/ using document root <none> and password file <none>
I20260619 14:59:50.535326 26375 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:50.535456 26375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:50.541955 26375 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260619 14:59:50.545329 26391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:50.547070 26375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:59:50.547235 26375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "4762805edd364b98b49497e3e5371ea1"
format_stamp: "Formatted at 2026-06-19 14:59:47 on dist-test-slave-76rk"
server_key: "620d875c17c0c0822ffc64bc6029a7ab"
server_key_iv: "b697a3b1155e050b436ba2bf1f195dd4"
server_key_version: "encryptionkey@0"
I20260619 14:59:50.548298 26375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:50.566788 26375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:50.571662 26375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:50.571916 26375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:50.592254 26375 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:42475
I20260619 14:59:50.592309 26442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:42475 every 8 connection(s)
I20260619 14:59:50.593912 26375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:59:50.600701 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 26375
I20260619 14:59:50.601523 26443 sys_catalog.cc:263] Verifying existing consensus state
I20260619 14:59:50.601542 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:36347
--local_ip_for_outbound_sockets=127.11.175.193
--tserver_master_addrs=127.11.175.254:42475
--webserver_port=33231
--webserver_interface=127.11.175.193
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:59:50.605034 26443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Bootstrap starting.
I20260619 14:59:50.631021 26443 log.cc:824] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:50.643891 26443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 14:59:50.644477 26443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Bootstrap complete.
I20260619 14:59:50.654222 26443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:50.655156 26443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4762805edd364b98b49497e3e5371ea1, State: Initialized, Role: FOLLOWER
I20260619 14:59:50.655778 26443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:50.656023 26443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:59:50.656152 26443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:59:50.656347 26443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 1 FOLLOWER]: Advancing to term 2
I20260619 14:59:50.660331 26443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:50.660888 26443 leader_election.cc:302] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4762805edd364b98b49497e3e5371ea1; no voters:
I20260619 14:59:50.661535 26443 leader_election.cc:288] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 14:59:50.661681 26447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 14:59:50.662981 26447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 2 LEADER]: Becoming Leader. State: Replica: 4762805edd364b98b49497e3e5371ea1, State: Running, Role: LEADER
I20260619 14:59:50.663604 26447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:50.664076 26443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:59:50.666217 26448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4762805edd364b98b49497e3e5371ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } } }
I20260619 14:59:50.666879 26448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:50.672128 26449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4762805edd364b98b49497e3e5371ea1. Latest consensus state: current_term: 2 leader_uuid: "4762805edd364b98b49497e3e5371ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } } }
I20260619 14:59:50.672823 26449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:50.675093 26453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:59:50.685096 26453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:59:50.687122 26453 catalog_manager.cc:1259] Loaded cluster ID: 92030b6128e44d118743f02960b33260
I20260619 14:59:50.687318 26453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 14:59:50.691550 26453 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 14:59:50.718475 26453 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 14:59:50.732576 26453 catalog_manager.cc:1504] Loading token signing keys...
I20260619 14:59:50.734596 26453 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Loaded TSK: 0
I20260619 14:59:50.736330 26453 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 14:59:50.977428 26445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:50.977715 26445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:50.977875 26445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:50.985399 26445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:50.985522 26445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:50.985705 26445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:59:50.995850 26445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:36347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=33231
--tserver_master_addrs=127.11.175.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:50.997319 26445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:50.998888 26445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:51.014524 26471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:51.015021 26473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:51.015445 26470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:51.016484 26445 server_base.cc:1034] running on GCE node
I20260619 14:59:51.017190 26445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:51.018319 26445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:51.019584 26445 hybrid_clock.cc:648] HybridClock initialized: now 1781881191019496 us; error 71 us; skew 500 ppm
I20260619 14:59:51.019971 26445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:51 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:51.029456 26445 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:59:51.031298 26445 webserver.cc:460] Webserver started at http://127.11.175.193:33231/ using document root <none> and password file <none>
I20260619 14:59:51.032075 26445 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:51.032243 26445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:51.038030 26445 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 14:59:51.041599 26480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:51.043325 26445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 14:59:51.043527 26445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "259102544c184ea6a3aa6252e81f3ca4"
format_stamp: "Formatted at 2026-06-19 14:59:48 on dist-test-slave-76rk"
server_key: "47a0cf780b6526361d43c75f0051382c"
server_key_iv: "9719d972aa2828867d7ac8863a00aa2a"
server_key_version: "encryptionkey@0"
I20260619 14:59:51.044729 26445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:51.066177 26445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:51.071111 26445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:51.071434 26445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:51.072539 26445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:51.074052 26445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:51.074196 26445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:51.074355 26445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:51.074448 26445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:51.114545 26445 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:36347
I20260619 14:59:51.114590 26592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:36347 every 8 connection(s)
I20260619 14:59:51.116256 26445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:59:51.122007 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 26445
I20260619 14:59:51.122694 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:41057
--local_ip_for_outbound_sockets=127.11.175.194
--tserver_master_addrs=127.11.175.254:42475
--webserver_port=44719
--webserver_interface=127.11.175.194
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 14:59:51 dist-test-slave-76rk krb5kdc[25825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:51.150921 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:51.120068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:55867)
0619 14:59:51.120896 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:51.120934 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:51.123289 (+ 2355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:51.125123 (+ 1834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:51.125135 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:51.125964 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:51.126834 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:51.126864 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:51.129526 (+ 2662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:51.129534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:51.132024 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:51.132045 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:51.133684 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:51.133694 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:51.134054 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:51.134992 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:51.135060 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:51.138687 (+ 3627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:51.144676 (+ 5989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:51.144691 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:51.144714 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:51.145522 (+ 808us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:51.146167 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:51.146174 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:51.146178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:51.146497 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:51.147668 (+ 1171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:51.147681 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:51.148279 (+ 598us) client_negotiation.cc:769] Sending connection context
0619 14:59:51.148693 (+ 414us) client_negotiation.cc:240] Negotiation successful
0619 14:59:51.149393 (+ 700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":863,"threads_started":1}
I20260619 14:59:51.151713 26596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:51.120862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:55867 (local address 127.11.175.254:42475)
0619 14:59:51.121533 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:51.121550 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:51.121613 (+ 63us) server_negotiation.cc:407] Connection header received
0619 14:59:51.124006 (+ 2393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:51.124029 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:51.124517 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:51.124909 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:51.127155 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:51.129246 (+ 2091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:51.132200 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:51.133394 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:51.133581 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:51.139101 (+ 5520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:51.139117 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:51.139144 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:51.139208 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:51.144439 (+ 5231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:51.145729 (+ 1290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:51.145752 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:51.145764 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:51.146016 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:51.146683 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:51.146690 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:51.146696 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:51.147491 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:51.147729 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:51.149205 (+ 1476us) server_negotiation.cc:299] Negotiation successful
0619 14:59:51.149875 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":1046,"threads_started":1}
I20260619 14:59:51.155637 26593 heartbeater.cc:347] Connected to a master server at 127.11.175.254:42475
I20260619 14:59:51.156032 26593 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:51.157114 26593 heartbeater.cc:511] Master 127.11.175.254:42475 requested a full tablet report, sending...
I20260619 14:59:51.159955 26408 ts_manager.cc:194] Registered new tserver with Master: 259102544c184ea6a3aa6252e81f3ca4 (127.11.175.193:36347)
I20260619 14:59:51.162343 26408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:55867
W20260619 14:59:51.464551 26597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:51.464917 26597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:51.465119 26597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:51.472782 26597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:51.472963 26597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:51.473141 26597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:59:51.483356 26597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:41057
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=44719
--tserver_master_addrs=127.11.175.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:51.484709 26597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:51.486053 26597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:51.499392 26602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:51.499387 26603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:51.499509 26605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:51.500636 26597 server_base.cc:1034] running on GCE node
I20260619 14:59:51.501343 26597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:51.502367 26597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:51.503597 26597 hybrid_clock.cc:648] HybridClock initialized: now 1781881191503497 us; error 94 us; skew 500 ppm
I20260619 14:59:51.503958 26597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:51 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:51.507278 26597 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:59:51.508744 26597 webserver.cc:460] Webserver started at http://127.11.175.194:44719/ using document root <none> and password file <none>
I20260619 14:59:51.509388 26597 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:51.509502 26597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:51.515014 26597 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 14:59:51.517776 26612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:51.519891 26597 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 14:59:51.520078 26597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "70f0f027df57489b8a09cad395699015"
format_stamp: "Formatted at 2026-06-19 14:59:48 on dist-test-slave-76rk"
server_key: "7a62622c53759265872ca4f37571c639"
server_key_iv: "fc168f780f24ff42a24fff57dfe4f472"
server_key_version: "encryptionkey@0"
I20260619 14:59:51.521588 26597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:51.539463 26597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:51.545547 26597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:51.545895 26597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:51.547164 26597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:51.549304 26597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:51.549453 26597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:51.549568 26597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:51.549644 26597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:51.586118 26597 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:41057
I20260619 14:59:51.586176 26724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:41057 every 8 connection(s)
I20260619 14:59:51.587751 26597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:59:51.591121 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 26597
I20260619 14:59:51.592072 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:41247
--local_ip_for_outbound_sockets=127.11.175.195
--tserver_master_addrs=127.11.175.254:42475
--webserver_port=45725
--webserver_interface=127.11.175.195
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 14:59:51 dist-test-slave-76rk krb5kdc[25825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:51.622159 26596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:51.590615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:40323 (local address 127.11.175.254:42475)
0619 14:59:51.590905 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:51.590910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:51.595522 (+ 4612us) server_negotiation.cc:407] Connection header received
0619 14:59:51.602897 (+ 7375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:51.602906 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:51.603034 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:51.603153 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:51.605292 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:51.606501 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:51.608956 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:51.609733 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:51.609888 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:51.614912 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:51.614920 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:51.614926 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:51.614970 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:51.618488 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:51.619688 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:51.619693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:51.619695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:51.619827 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:51.620385 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:51.620389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:51.620391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:51.620690 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:51.620825 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:51.621724 (+ 899us) server_negotiation.cc:299] Negotiation successful
0619 14:59:51.621917 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260619 14:59:51.623593 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:51.591344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:40323)
0619 14:59:51.595318 (+ 3974us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:51.595381 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:51.602414 (+ 7033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:51.603476 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:51.603490 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:51.604282 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:51.605071 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:51.605096 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:51.606748 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:51.606756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:51.608783 (+ 2027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:51.608798 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:51.609995 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:51.610003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:51.610301 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:51.611158 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:51.611189 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:51.614578 (+ 3389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:51.618698 (+ 4120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:51.618712 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:51.618734 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:51.619515 (+ 781us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:51.619951 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:51.619958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:51.619965 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:51.620248 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:51.620837 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:51.620845 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:51.621301 (+ 456us) client_negotiation.cc:769] Sending connection context
0619 14:59:51.621619 (+ 318us) client_negotiation.cc:240] Negotiation successful
0619 14:59:51.622186 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3589,"thread_start_us":3354,"threads_started":1}
I20260619 14:59:51.625149 26725 heartbeater.cc:347] Connected to a master server at 127.11.175.254:42475
I20260619 14:59:51.625586 26725 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:51.626408 26725 heartbeater.cc:511] Master 127.11.175.254:42475 requested a full tablet report, sending...
I20260619 14:59:51.628032 26408 ts_manager.cc:194] Registered new tserver with Master: 70f0f027df57489b8a09cad395699015 (127.11.175.194:41057)
I20260619 14:59:51.628830 26408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:40323
W20260619 14:59:51.935359 26728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:51.935719 26728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:51.935907 26728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:51.943318 26728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:51.943490 26728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:51.943687 26728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:59:51.953843 26728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:41247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=45725
--tserver_master_addrs=127.11.175.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:51.955252 26728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:51.956626 26728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:51.970249 26736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:51.970275 26734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:51.970584 26733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:51.971747 26728 server_base.cc:1034] running on GCE node
I20260619 14:59:51.972455 26728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:51.973517 26728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:51.974798 26728 hybrid_clock.cc:648] HybridClock initialized: now 1781881191974743 us; error 58 us; skew 500 ppm
I20260619 14:59:51.975162 26728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:51 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:51.978473 26728 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:59:51.980062 26728 webserver.cc:460] Webserver started at http://127.11.175.195:45725/ using document root <none> and password file <none>
I20260619 14:59:51.980772 26728 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:51.980914 26728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:51.986409 26728 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260619 14:59:51.989298 26743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:51.990892 26728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:59:51.991071 26728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "fab9a33b44db47808f10e7e8407d63c3"
format_stamp: "Formatted at 2026-06-19 14:59:49 on dist-test-slave-76rk"
server_key: "2468343ff45c9f7af3d5403cfc2e7fb2"
server_key_iv: "daff209918535d4bb8197c0b9263b140"
server_key_version: "encryptionkey@0"
I20260619 14:59:51.992063 26728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:52.014195 26728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:52.018915 26728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:52.019202 26728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:52.020231 26728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:52.021696 26728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:52.021783 26728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:52.021915 26728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:52.022032 26728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:52.057268 26728 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:41247
I20260619 14:59:52.057323 26855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:41247 every 8 connection(s)
I20260619 14:59:52.058746 26728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:59:52.062081 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 26728
Jun 19 14:59:52 dist-test-slave-76rk krb5kdc[25825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 14:59:52.082840 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.063036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54710 (local address 127.11.175.254:42475)
0619 14:59:52.063949 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.064007 (+ 58us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.065172 (+ 1165us) server_negotiation.cc:407] Connection header received
0619 14:59:52.065267 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.065272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.065352 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.065459 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:52.067869 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.068841 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.070787 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.071293 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.071405 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.075272 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:52.075281 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:52.075288 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:52.075339 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:52.079716 (+ 4377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:52.080524 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:52.080536 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:52.080540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:52.080706 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:52.081204 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:52.081208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:52.081211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:52.081518 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:52.081876 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.082136 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.082345 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":815,"threads_started":1}
I20260619 14:59:52.084362 26596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.061596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:44031 (local address 127.11.175.254:42475)
0619 14:59:52.061839 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.061847 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.063296 (+ 1449us) server_negotiation.cc:407] Connection header received
0619 14:59:52.066174 (+ 2878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.066178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.066238 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.066340 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:52.068462 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.069345 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.071494 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.072111 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.072262 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.076981 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:52.076989 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:52.076994 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:52.077055 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:52.080818 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:52.081694 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:52.081699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:52.081702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:52.081852 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:52.082390 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:52.082395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:52.082398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:52.082756 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:52.082923 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.083794 (+ 871us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.084009 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 14:59:52.085521 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.062141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:44031)
0619 14:59:52.063126 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.063175 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:52.065823 (+ 2648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:52.066701 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:52.066717 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:52.067546 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0619 14:59:52.068279 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.068298 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.069527 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.069533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.071313 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.071325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.072449 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.072459 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.072794 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.073573 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 14:59:52.073621 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0619 14:59:52.076695 (+ 3074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 14:59:52.080974 (+ 4279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:52.080985 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:52.081005 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:52.081526 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:52.081984 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 14:59:52.081992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 14:59:52.081995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 14:59:52.082251 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 14:59:52.082974 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 14:59:52.082986 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 14:59:52.083404 (+ 418us) client_negotiation.cc:769] Sending connection context
0619 14:59:52.083688 (+ 284us) client_negotiation.cc:240] Negotiation successful
0619 14:59:52.084271 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":411,"threads_started":1}
I20260619 14:59:52.087175 26856 heartbeater.cc:347] Connected to a master server at 127.11.175.254:42475
I20260619 14:59:52.087534 26856 heartbeater.cc:464] Registering TS with master...
I20260619 14:59:52.088348 26856 heartbeater.cc:511] Master 127.11.175.254:42475 requested a full tablet report, sending...
I20260619 14:59:52.090196 26408 ts_manager.cc:194] Registered new tserver with Master: fab9a33b44db47808f10e7e8407d63c3 (127.11.175.195:41247)
I20260619 14:59:52.091153 26408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:44031
I20260619 14:59:52.098585 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 14:59:52.106537 26596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.099708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.11.175.254:42475)
0619 14:59:52.099924 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.099929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.100469 (+ 540us) server_negotiation.cc:407] Connection header received
0619 14:59:52.100667 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.100671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.100916 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.101064 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:59:52.101539 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.102294 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.104154 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.104603 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.104715 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.105349 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:59:52.105891 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:59:52.106009 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.106191 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.106284 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260619 14:59:52.107699 26408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 14:59:52.115185 26408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 14:59:52.149489 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.128507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.11.175.193:36347)
0619 14:59:52.139574 (+ 11067us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.139589 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.140607 (+ 1018us) server_negotiation.cc:407] Connection header received
0619 14:59:52.140972 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.140983 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.141275 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.141474 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:52.142689 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.144262 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.147063 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.148148 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.148378 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.148565 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.148917 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.149095 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10799,"thread_start_us":11094,"threads_started":1}
I20260619 14:59:52.150345 26870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.128285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.0.0.1:41990)
0619 14:59:52.140399 (+ 12114us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.140475 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:52.140699 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:52.141906 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:52.141918 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:52.141959 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:52.142413 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.142429 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.144465 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.144473 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.146907 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.146920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.148388 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.148397 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.148582 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.148647 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 14:59:52.149724 (+ 1077us) client_negotiation.cc:240] Negotiation successful
0619 14:59:52.149994 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11811,"thread_start_us":11718,"threads_started":1}
I20260619 14:59:52.153223 26869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.129596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34884 (local address 127.11.175.195:41247)
0619 14:59:52.143294 (+ 13698us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.143306 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.143352 (+ 46us) server_negotiation.cc:407] Connection header received
0619 14:59:52.143463 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.143471 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.143775 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.143997 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:52.144601 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.146247 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.149990 (+ 3743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.151608 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.151829 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.152049 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.152654 (+ 605us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.152824 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13489,"thread_start_us":13313,"threads_started":1}
I20260619 14:59:52.153735 26866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.128772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:41247 (local address 127.0.0.1:34884)
0619 14:59:52.140399 (+ 11627us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.140480 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:52.140699 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:52.144000 (+ 3301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:52.144005 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:52.144031 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:52.144361 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.144374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.146408 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.146415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.149542 (+ 3127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.149558 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.152213 (+ 2655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.152219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.152435 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.152478 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 14:59:52.152581 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 14:59:52.152738 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11390,"mutex_wait_us":23,"thread_start_us":11153,"threads_started":1}
I20260619 14:59:52.153906 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.130256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:41057 (local address 127.0.0.1:45258)
0619 14:59:52.141308 (+ 11052us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.141345 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:52.141457 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:52.144548 (+ 3091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:52.144553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:52.144591 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:52.145159 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.145173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.147103 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.147109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.149856 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.149875 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.151738 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.151746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.151932 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.151967 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 14:59:52.152057 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 14:59:52.152241 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10902,"mutex_wait_us":291,"thread_start_us":10728,"threads_started":1}
I20260619 14:59:52.154690 26528 tablet_service.cc:1467] Processing CreateTablet for tablet 328c083a793b492897c3a0a793eb8244 (DEFAULT_TABLE table=test-table [id=8dba7919f11345b5bdef506756b45aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:59:52.156417 26528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 328c083a793b492897c3a0a793eb8244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:52.157440 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.129441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.11.175.194:41057)
0619 14:59:52.143630 (+ 14189us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.143639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.143667 (+ 28us) server_negotiation.cc:407] Connection header received
0619 14:59:52.143771 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.143779 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.144017 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.144228 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:52.145389 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.146924 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.150032 (+ 3108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.151506 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.156818 (+ 5312us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.156963 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.157036 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.157185 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14023,"thread_start_us":13874,"threads_started":1}
I20260619 14:59:52.157722 26791 tablet_service.cc:1467] Processing CreateTablet for tablet 328c083a793b492897c3a0a793eb8244 (DEFAULT_TABLE table=test-table [id=8dba7919f11345b5bdef506756b45aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:59:52.159188 26791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 328c083a793b492897c3a0a793eb8244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:52.160482 26660 tablet_service.cc:1467] Processing CreateTablet for tablet 328c083a793b492897c3a0a793eb8244 (DEFAULT_TABLE table=test-table [id=8dba7919f11345b5bdef506756b45aa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 14:59:52.161903 26660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 328c083a793b492897c3a0a793eb8244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 14:59:52.167711 26593 heartbeater.cc:503] Master 127.11.175.254:42475 was elected leader, sending a full tablet report...
I20260619 14:59:52.173396 26872 tablet_bootstrap.cc:492] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: Bootstrap starting.
I20260619 14:59:52.176533 26872 tablet_bootstrap.cc:654] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:52.177554 26872 log.cc:824] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:52.178537 26873 tablet_bootstrap.cc:492] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Bootstrap starting.
I20260619 14:59:52.180891 26875 tablet_bootstrap.cc:492] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: Bootstrap starting.
I20260619 14:59:52.182104 26872 tablet_bootstrap.cc:492] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: No bootstrap required, opened a new log
I20260619 14:59:52.182375 26872 ts_tablet_manager.cc:1397] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260619 14:59:52.182686 26873 tablet_bootstrap.cc:654] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:52.184100 26875 tablet_bootstrap.cc:654] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: Neither blocks nor log segments found. Creating new log.
I20260619 14:59:52.184211 26873 log.cc:824] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:52.185278 26875 log.cc:824] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:52.188159 26875 tablet_bootstrap.cc:492] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: No bootstrap required, opened a new log
I20260619 14:59:52.188449 26875 ts_tablet_manager.cc:1397] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260619 14:59:52.188686 26873 tablet_bootstrap.cc:492] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: No bootstrap required, opened a new log
I20260619 14:59:52.188946 26873 ts_tablet_manager.cc:1397] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260619 14:59:52.193499 26872 raft_consensus.cc:348] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } }
I20260619 14:59:52.193994 26872 raft_consensus.cc:374] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:52.194115 26872 raft_consensus.cc:729] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab9a33b44db47808f10e7e8407d63c3, State: Initialized, Role: FOLLOWER
I20260619 14:59:52.194845 26872 consensus_queue.cc:260] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } }
I20260619 14:59:52.196759 26856 heartbeater.cc:503] Master 127.11.175.254:42475 was elected leader, sending a full tablet report...
I20260619 14:59:52.197180 26875 raft_consensus.cc:348] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } }
I20260619 14:59:52.197523 26872 ts_tablet_manager.cc:1428] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260619 14:59:52.197613 26875 raft_consensus.cc:374] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:52.197743 26875 raft_consensus.cc:729] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70f0f027df57489b8a09cad395699015, State: Initialized, Role: FOLLOWER
I20260619 14:59:52.198446 26875 consensus_queue.cc:260] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } }
I20260619 14:59:52.199661 26725 heartbeater.cc:503] Master 127.11.175.254:42475 was elected leader, sending a full tablet report...
I20260619 14:59:52.200393 26875 ts_tablet_manager.cc:1428] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: Time spent starting tablet: real 0.012s user 0.006s sys 0.004s
I20260619 14:59:52.202432 26873 raft_consensus.cc:348] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } }
I20260619 14:59:52.202920 26873 raft_consensus.cc:374] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 14:59:52.203059 26873 raft_consensus.cc:729] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 259102544c184ea6a3aa6252e81f3ca4, State: Initialized, Role: FOLLOWER
I20260619 14:59:52.203667 26873 consensus_queue.cc:260] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } }
I20260619 14:59:52.205250 26873 ts_tablet_manager.cc:1428] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260619 14:59:52.310593 26857 tablet.cc:2369] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 14:59:52.339991 26726 tablet.cc:2369] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 14:59:52.369112 26594 tablet.cc:2369] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 14:59:52.385694 26880 raft_consensus.cc:482] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 14:59:52.386016 26880 raft_consensus.cc:504] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } }
I20260619 14:59:52.387822 26880 leader_election.cc:288] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fab9a33b44db47808f10e7e8407d63c3 (127.11.175.195:41247), 70f0f027df57489b8a09cad395699015 (127.11.175.194:41057)
I20260619 14:59:52.394742 26869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.388229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:55073 (local address 127.11.175.195:41247)
0619 14:59:52.388451 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.388458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.389049 (+ 591us) server_negotiation.cc:407] Connection header received
0619 14:59:52.389290 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.389296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.389365 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.389466 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:52.390041 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.390833 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.392896 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.393710 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.393915 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.393997 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.394283 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.394426 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 14:59:52.394919 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.388179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:41247 (local address 127.11.175.193:55073)
0619 14:59:52.388928 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.388944 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:52.389131 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:52.389494 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:52.389499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:52.389535 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:52.389895 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.389905 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.390967 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.390975 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.392680 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.392697 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.393945 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.393952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.394111 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.394137 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 14:59:52.394258 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 14:59:52.394408 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":3,"thread_start_us":435,"threads_started":1}
I20260619 14:59:52.395090 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.388241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:48529 (local address 127.11.175.194:41057)
0619 14:59:52.388451 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.388458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.389608 (+ 1150us) server_negotiation.cc:407] Connection header received
0619 14:59:52.389865 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.389869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.389927 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.390026 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 14:59:52.390671 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.391422 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.393225 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.394036 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.394218 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.394300 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.394595 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.394760 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 14:59:52.395325 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.388458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:41057 (local address 127.11.175.193:48529)
0619 14:59:52.389483 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.389497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 14:59:52.389693 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 14:59:52.390042 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 14:59:52.390046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 14:59:52.390069 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 14:59:52.390458 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.390469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.391566 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.391572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.393054 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 14:59:52.393067 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.394280 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.394286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 14:59:52.394432 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.394455 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 14:59:52.394569 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 14:59:52.394746 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":2,"thread_start_us":660,"threads_started":1}
I20260619 14:59:52.396231 26811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "328c083a793b492897c3a0a793eb8244" candidate_uuid: "259102544c184ea6a3aa6252e81f3ca4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab9a33b44db47808f10e7e8407d63c3" is_pre_election: true
I20260619 14:59:52.396391 26680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "328c083a793b492897c3a0a793eb8244" candidate_uuid: "259102544c184ea6a3aa6252e81f3ca4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70f0f027df57489b8a09cad395699015" is_pre_election: true
I20260619 14:59:52.396924 26680 raft_consensus.cc:2454] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 259102544c184ea6a3aa6252e81f3ca4 in term 0.
I20260619 14:59:52.396903 26811 raft_consensus.cc:2454] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 259102544c184ea6a3aa6252e81f3ca4 in term 0.
I20260619 14:59:52.397704 26483 leader_election.cc:302] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 259102544c184ea6a3aa6252e81f3ca4, 70f0f027df57489b8a09cad395699015, fab9a33b44db47808f10e7e8407d63c3; no voters:
I20260619 14:59:52.398132 26880 raft_consensus.cc:2781] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 14:59:52.398298 26880 raft_consensus.cc:482] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 14:59:52.398413 26880 raft_consensus.cc:3037] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:52.400934 26880 raft_consensus.cc:504] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } }
I20260619 14:59:52.401635 26880 leader_election.cc:288] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [CANDIDATE]: Term 1 election: Requested vote from peers fab9a33b44db47808f10e7e8407d63c3 (127.11.175.195:41247), 70f0f027df57489b8a09cad395699015 (127.11.175.194:41057)
I20260619 14:59:52.402212 26680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "328c083a793b492897c3a0a793eb8244" candidate_uuid: "259102544c184ea6a3aa6252e81f3ca4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70f0f027df57489b8a09cad395699015"
I20260619 14:59:52.402220 26811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "328c083a793b492897c3a0a793eb8244" candidate_uuid: "259102544c184ea6a3aa6252e81f3ca4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab9a33b44db47808f10e7e8407d63c3"
I20260619 14:59:52.402427 26680 raft_consensus.cc:3037] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:52.402446 26811 raft_consensus.cc:3037] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 14:59:52.404724 26811 raft_consensus.cc:2454] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 259102544c184ea6a3aa6252e81f3ca4 in term 1.
I20260619 14:59:52.404731 26680 raft_consensus.cc:2454] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 259102544c184ea6a3aa6252e81f3ca4 in term 1.
I20260619 14:59:52.405241 26483 leader_election.cc:302] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 259102544c184ea6a3aa6252e81f3ca4, fab9a33b44db47808f10e7e8407d63c3; no voters:
I20260619 14:59:52.405599 26880 raft_consensus.cc:2781] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 14:59:52.406129 26880 raft_consensus.cc:686] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 1 LEADER]: Becoming Leader. State: Replica: 259102544c184ea6a3aa6252e81f3ca4, State: Running, Role: LEADER
I20260619 14:59:52.406749 26880 consensus_queue.cc:237] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } }
I20260619 14:59:52.411846 26407 catalog_manager.cc:5495] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 259102544c184ea6a3aa6252e81f3ca4 (127.11.175.193). New cstate: current_term: 1 leader_uuid: "259102544c184ea6a3aa6252e81f3ca4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259102544c184ea6a3aa6252e81f3ca4" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260619 14:59:52.455376 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.449447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41994 (local address 127.11.175.193:36347)
0619 14:59:52.449670 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.449677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.449699 (+ 22us) server_negotiation.cc:407] Connection header received
0619 14:59:52.449973 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.449978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.450057 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.450230 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 14:59:52.450621 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.451372 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.452923 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.453351 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.453493 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.454047 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 14:59:52.454748 (+ 701us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 14:59:52.454914 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.455013 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.455100 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 14:59:52.463989 26680 raft_consensus.cc:1261] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015 [term 1 FOLLOWER]: Refusing update from remote peer 259102544c184ea6a3aa6252e81f3ca4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:59:52.463990 26811 raft_consensus.cc:1261] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3 [term 1 FOLLOWER]: Refusing update from remote peer 259102544c184ea6a3aa6252e81f3ca4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 14:59:52.465117 26883 consensus_queue.cc:1035] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70f0f027df57489b8a09cad395699015" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:59:52.465569 26880 consensus_queue.cc:1035] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fab9a33b44db47808f10e7e8407d63c3" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 41247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 14:59:52.479242 26887 mvcc.cc:205] Tried to move back new op lower bound from 7298585364321697792 to 7298585364102131712. Current Snapshot: MvccSnapshot[applied={T|T < 7298585364321697792}]
I20260619 14:59:52.495690 26407 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:59:52.495991 26407 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 14:59:52.499904 26407 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Sending DeleteTablet for 3 replicas of tablet 328c083a793b492897c3a0a793eb8244
I20260619 14:59:52.501122 26791 tablet_service.cc:1514] Processing DeleteTablet for tablet 328c083a793b492897c3a0a793eb8244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:59:52 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:34884
I20260619 14:59:52.501353 26660 tablet_service.cc:1514] Processing DeleteTablet for tablet 328c083a793b492897c3a0a793eb8244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:59:52 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:45258
I20260619 14:59:52.501523 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 26445
I20260619 14:59:52.502132 26528 tablet_service.cc:1514] Processing DeleteTablet for tablet 328c083a793b492897c3a0a793eb8244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 14:59:52 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:41990
I20260619 14:59:52.502957 26897 tablet_replica.cc:330] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: stopping tablet replica
I20260619 14:59:52.503047 26896 tablet_replica.cc:330] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: stopping tablet replica
I20260619 14:59:52.503384 26897 raft_consensus.cc:2229] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:59:52.503394 26896 raft_consensus.cc:2229] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 14:59:52.503878 26896 raft_consensus.cc:2258] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:59:52.503880 26897 raft_consensus.cc:2258] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:59:52.505084 26898 tablet_replica.cc:330] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: stopping tablet replica
I20260619 14:59:52.505511 26898 raft_consensus.cc:2229] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 14:59:52.506269 26898 raft_consensus.cc:2258] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 14:59:52.506505 26896 ts_tablet_manager.cc:1905] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:59:52.507894 26897 ts_tablet_manager.cc:1905] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:59:52.513531 26896 ts_tablet_manager.cc:1918] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:59:52.513710 26896 log.cc:1196] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/328c083a793b492897c3a0a793eb8244
I20260619 14:59:52.514225 26896 ts_tablet_manager.cc:1939] T 328c083a793b492897c3a0a793eb8244 P fab9a33b44db47808f10e7e8407d63c3: Deleting consensus metadata
I20260619 14:59:52.515126 26897 ts_tablet_manager.cc:1918] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:59:52.515316 26897 log.cc:1196] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/328c083a793b492897c3a0a793eb8244
I20260619 14:59:52.515877 26897 ts_tablet_manager.cc:1939] T 328c083a793b492897c3a0a793eb8244 P 70f0f027df57489b8a09cad395699015: Deleting consensus metadata
I20260619 14:59:52.516294 26394 catalog_manager.cc:4841] TS fab9a33b44db47808f10e7e8407d63c3 (127.11.175.195:41247): tablet 328c083a793b492897c3a0a793eb8244 (table test-table [id=8dba7919f11345b5bdef506756b45aa9]) successfully deleted
I20260619 14:59:52.517611 26392 catalog_manager.cc:4841] TS 70f0f027df57489b8a09cad395699015 (127.11.175.194:41057): tablet 328c083a793b492897c3a0a793eb8244 (table test-table [id=8dba7919f11345b5bdef506756b45aa9]) successfully deleted
I20260619 14:59:52.520087 26898 ts_tablet_manager.cc:1905] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:59:52.524257 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.503311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.11.175.193:36347)
0619 14:59:52.503935 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.503941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.503965 (+ 24us) server_negotiation.cc:407] Connection header received
0619 14:59:52.504022 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.504027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.504109 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.504215 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:52.506401 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.507682 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.510048 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.511283 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.511412 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.515705 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:52.515723 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:52.515744 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:52.515803 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:52.520231 (+ 4428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:52.521503 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:52.521517 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:52.521521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:52.521712 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:52.522184 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:52.522190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:52.522194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:52.522703 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:52.523019 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.523362 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.523637 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483}
I20260619 14:59:52.525100 26588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:52.630405 26898 ts_tablet_manager.cc:1918] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:59:52.630699 26898 log.cc:1196] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/328c083a793b492897c3a0a793eb8244
I20260619 14:59:52.630991 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 26445
I20260619 14:59:52.650501 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 26597
W20260619 14:59:52.650604 26395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.175.193:36347 (error 108)
I20260619 14:59:52.652343 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.651922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.0.0.1:42006)
0619 14:59:52.652091 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.652207 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 14:59:52.652663 26395 catalog_manager.cc:4584] TS 259102544c184ea6a3aa6252e81f3ca4 (127.11.175.193:36347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 328c083a793b492897c3a0a793eb8244: Network error: Client connection negotiation failed: client connection to 127.11.175.193:36347: connect: Connection refused (error 111)
I20260619 14:59:52.666538 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.651413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45270 (local address 127.11.175.194:41057)
0619 14:59:52.651624 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.651632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.651908 (+ 276us) server_negotiation.cc:407] Connection header received
0619 14:59:52.652188 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.652195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.652287 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.652410 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:52.654217 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.655206 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.656677 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.657095 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.657257 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.660208 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:52.660224 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:52.660241 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:52.660280 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:52.663502 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:52.664334 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:52.664342 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:52.664344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:52.664470 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:52.664875 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:52.664881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:52.664884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:52.665338 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:52.665633 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.665964 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.666191 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 14:59:52.667243 26720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:52.706336 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.705779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.0.0.1:42018)
0619 14:59:52.706076 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.706189 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260619 14:59:52.707406 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.707055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.0.0.1:42024)
0619 14:59:52.707203 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.707280 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 14:59:52.759152 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 26597
I20260619 14:59:52.768282 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.767639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.0.0.1:42026)
0619 14:59:52.767925 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.768045 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260619 14:59:52.769559 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.769102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.0.0.1:42032)
0619 14:59:52.769290 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.769389 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 14:59:52.778038 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 26728
I20260619 14:59:52.793087 26869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.778925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34894 (local address 127.11.175.195:41247)
0619 14:59:52.779152 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 14:59:52.779158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 14:59:52.779256 (+ 98us) server_negotiation.cc:407] Connection header received
0619 14:59:52.779463 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 14:59:52.779469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 14:59:52.779544 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 14:59:52.779640 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 14:59:52.781122 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.781909 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.783475 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 14:59:52.783898 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 14:59:52.784007 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 14:59:52.787514 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 14:59:52.787531 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 14:59:52.787546 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 14:59:52.787586 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 14:59:52.790145 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:52.790964 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:52.790973 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:52.790977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:52.791137 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 14:59:52.791529 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 14:59:52.791534 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 14:59:52.791537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 14:59:52.791883 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 14:59:52.792185 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0619 14:59:52.792509 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 14:59:52.792746 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 14:59:52.793862 26851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:52.850226 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.849596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.0.0.1:42046)
0619 14:59:52.849872 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.849991 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260619 14:59:52.851507 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:52.851075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.0.0.1:42054)
0619 14:59:52.851290 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:52.851378 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260619 14:59:52.888509 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 26728
I20260619 14:59:52.908068 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 26375
I20260619 14:59:52.909332 26438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 14:59:53.026147 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:53.025631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36347 (local address 127.0.0.1:42056)
0619 14:59:53.025831 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:53.025919 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 14:59:53.026149 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 26375
I20260619 14:59:53.048764 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:42475
--webserver_interface=127.11.175.254
--webserver_port=37253
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:42475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 14:59:53.384366 26904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:53.384694 26904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:53.384858 26904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:53.392402 26904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 14:59:53.392518 26904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:53.392624 26904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:53.392724 26904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 14:59:53.392810 26904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 14:59:53.402992 26904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:42475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:42475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=37253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:53.404495 26904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:53.405825 26904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:53.418197 26912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:53.418293 26910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:53.418467 26909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:53.419268 26904 server_base.cc:1034] running on GCE node
I20260619 14:59:53.420111 26904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:53.421594 26904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:53.422861 26904 hybrid_clock.cc:648] HybridClock initialized: now 1781881193422784 us; error 47 us; skew 500 ppm
I20260619 14:59:53.423234 26904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:53 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881193, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:53.426501 26904 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 14:59:53.428296 26904 webserver.cc:460] Webserver started at http://127.11.175.254:37253/ using document root <none> and password file <none>
I20260619 14:59:53.429013 26904 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:53.429162 26904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:53.434779 26904 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 14:59:53.437714 26919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:53.439199 26904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:59:53.439404 26904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "4762805edd364b98b49497e3e5371ea1"
format_stamp: "Formatted at 2026-06-19 14:59:47 on dist-test-slave-76rk"
server_key: "620d875c17c0c0822ffc64bc6029a7ab"
server_key_iv: "b697a3b1155e050b436ba2bf1f195dd4"
server_key_version: "encryptionkey@0"
I20260619 14:59:53.440439 26904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:53.465173 26904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:53.469413 26904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:53.469703 26904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:53.489415 26904 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:42475
I20260619 14:59:53.489455 26970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:42475 every 8 connection(s)
I20260619 14:59:53.491024 26904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 14:59:53.493383 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 26904
I20260619 14:59:53.494170 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:36347
--local_ip_for_outbound_sockets=127.11.175.193
--tserver_master_addrs=127.11.175.254:42475
--webserver_port=33231
--webserver_interface=127.11.175.193
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:59:53.498513 26971 sys_catalog.cc:263] Verifying existing consensus state
I20260619 14:59:53.503146 26971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Bootstrap starting.
I20260619 14:59:53.534039 26971 log.cc:824] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Log is configured to *not* fsync() on all Append() calls
I20260619 14:59:53.556463 26971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 14:59:53.557215 26971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1: Bootstrap complete.
I20260619 14:59:53.572463 26971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:53.573734 26971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4762805edd364b98b49497e3e5371ea1, State: Initialized, Role: FOLLOWER
I20260619 14:59:53.574496 26971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:53.574916 26971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 14:59:53.575066 26971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 14:59:53.575258 26971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 2 FOLLOWER]: Advancing to term 3
I20260619 14:59:53.579345 26971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:53.579859 26971 leader_election.cc:302] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4762805edd364b98b49497e3e5371ea1; no voters:
I20260619 14:59:53.580468 26971 leader_election.cc:288] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 14:59:53.580668 26975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 14:59:53.581849 26975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [term 3 LEADER]: Becoming Leader. State: Replica: 4762805edd364b98b49497e3e5371ea1, State: Running, Role: LEADER
I20260619 14:59:53.582538 26971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 14:59:53.582477 26975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } }
I20260619 14:59:53.586705 26976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4762805edd364b98b49497e3e5371ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } } }
I20260619 14:59:53.587096 26976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:53.587643 26977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4762805edd364b98b49497e3e5371ea1. Latest consensus state: current_term: 3 leader_uuid: "4762805edd364b98b49497e3e5371ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4762805edd364b98b49497e3e5371ea1" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42475 } } }
I20260619 14:59:53.587936 26977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4762805edd364b98b49497e3e5371ea1 [sys.catalog]: This master's current role is: LEADER
I20260619 14:59:53.595955 26986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 14:59:53.604866 26986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 14:59:53.606441 26986 catalog_manager.cc:1259] Loaded cluster ID: 92030b6128e44d118743f02960b33260
I20260619 14:59:53.606546 26986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 14:59:53.608634 26986 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd1c10389ad google::LogMessage::Fail() at ??:0
@ 0x7fd1c103b88a google::LogMessage::SendToLog() at ??:0
@ 0x7fd1c10384de google::LogMessage::Flush() at ??:0
@ 0x7fd1c1039a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd1d6a6b512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd1d6a68976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd1d6af0610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd1d6af0439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd1c3c84345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd1c3c8791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd1c3c87799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd1c3c62eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd1c6f3c6db start_thread at ??:0
@ 0x7fd1bfed261f clone at ??:0
W20260619 14:59:53.915313 26973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:53.915607 26973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:53.915784 26973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:53.924109 26973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:53.924248 26973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:53.924434 26973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 14:59:53.934616 26973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:36347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=33231
--tserver_master_addrs=127.11.175.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:53.936103 26973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:53.937482 26973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:53.952019 27014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:53.952001 27016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:53.952281 27013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:53.952870 26973 server_base.cc:1034] running on GCE node
I20260619 14:59:53.953516 26973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:53.954493 26973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:53.955744 26973 hybrid_clock.cc:648] HybridClock initialized: now 1781881193955678 us; error 75 us; skew 500 ppm
I20260619 14:59:53.956116 26973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:53 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881193, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:53.959712 26973 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 14:59:53.961289 26973 webserver.cc:460] Webserver started at http://127.11.175.193:33231/ using document root <none> and password file <none>
I20260619 14:59:53.961969 26973 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:53.962074 26973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:53.967631 26973 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 14:59:53.970350 27023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:53.972126 26973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 14:59:53.972295 26973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "259102544c184ea6a3aa6252e81f3ca4"
format_stamp: "Formatted at 2026-06-19 14:59:48 on dist-test-slave-76rk"
server_key: "47a0cf780b6526361d43c75f0051382c"
server_key_iv: "9719d972aa2828867d7ac8863a00aa2a"
server_key_version: "encryptionkey@0"
I20260619 14:59:53.973290 26973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:54.013964 26973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:54.018783 26973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:54.019023 26973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:54.020013 26973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:54.021967 27030 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260619 14:59:54.027626 27030 ts_tablet_manager.cc:1852] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260619 14:59:54.028646 27030 ts_tablet_manager.cc:1905] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 14:59:54.036114 27030 ts_tablet_manager.cc:1918] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 14:59:54.036463 27030 ts_tablet_manager.cc:1939] T 328c083a793b492897c3a0a793eb8244 P 259102544c184ea6a3aa6252e81f3ca4: Deleting consensus metadata
I20260619 14:59:54.036935 26973 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260619 14:59:54.037107 26973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260619 14:59:54.037267 26973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:54.037365 26973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:54.073683 26973 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:36347
I20260619 14:59:54.073746 27136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:36347 every 8 connection(s)
I20260619 14:59:54.075167 26973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 14:59:54.078356 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 26973
I20260619 14:59:54.079054 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:41057
--local_ip_for_outbound_sockets=127.11.175.194
--tserver_master_addrs=127.11.175.254:42475
--webserver_port=44719
--webserver_interface=127.11.175.194
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:59:54.080655 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:54.078497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:44339)
0619 14:59:54.079462 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:54.079770 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":373,"threads_started":1}
W20260619 14:59:54.081267 27137 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:42475 (0 consecutive failures): Network error: Failed to ping master at 127.11.175.254:42475: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
I20260619 14:59:54.082207 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:54.081858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:53171)
0619 14:59:54.082036 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:54.082106 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260619 14:59:54.083185 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:54.082875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:56067)
0619 14:59:54.083001 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:54.083071 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 14:59:54.083473 27137 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 14:59:54.410480 27140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:54.410804 27140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:54.410980 27140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:54.418391 27140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:54.418517 27140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:54.418736 27140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 14:59:54.428696 27140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:41057
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=44719
--tserver_master_addrs=127.11.175.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:54.430042 27140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:54.431404 27140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:54.444670 27148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:54.444871 27145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:54.444687 27146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:54.445863 27140 server_base.cc:1034] running on GCE node
I20260619 14:59:54.446516 27140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:54.447598 27140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:54.448894 27140 hybrid_clock.cc:648] HybridClock initialized: now 1781881194448845 us; error 36 us; skew 500 ppm
I20260619 14:59:54.449278 27140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:54 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881194, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:54.452585 27140 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 14:59:54.454156 27140 webserver.cc:460] Webserver started at http://127.11.175.194:44719/ using document root <none> and password file <none>
I20260619 14:59:54.454952 27140 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:54.455104 27140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:54.460601 27140 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 14:59:54.463670 27155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:54.465128 27140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:59:54.465312 27140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "70f0f027df57489b8a09cad395699015"
format_stamp: "Formatted at 2026-06-19 14:59:48 on dist-test-slave-76rk"
server_key: "7a62622c53759265872ca4f37571c639"
server_key_iv: "fc168f780f24ff42a24fff57dfe4f472"
server_key_version: "encryptionkey@0"
I20260619 14:59:54.466317 27140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:54.486397 27140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:54.491067 27140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:54.491384 27140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:54.492460 27140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:54.493876 27140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:54.494012 27140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:54.494156 27140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:54.494254 27140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:54.529665 27140 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:41057
I20260619 14:59:54.529829 27267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:41057 every 8 connection(s)
I20260619 14:59:54.531246 27140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 14:59:54.534392 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 27140
I20260619 14:59:54.535225 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:41247
--local_ip_for_outbound_sockets=127.11.175.195
--tserver_master_addrs=127.11.175.254:42475
--webserver_port=45725
--webserver_interface=127.11.175.195
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 14:59:54.536464 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:54.534359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:49643)
0619 14:59:54.535236 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:54.535536 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":347,"threads_started":1}
W20260619 14:59:54.537197 27268 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:42475 (0 consecutive failures): Network error: Failed to ping master at 127.11.175.254:42475: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
I20260619 14:59:54.538205 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:54.537830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:56767)
0619 14:59:54.538009 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:54.538094 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 14:59:54.539173 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:54.538821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:49887)
0619 14:59:54.538970 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:54.539059 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 14:59:54.539438 27268 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 14:59:54.884037 27271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 14:59:54.884373 27271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 14:59:54.884554 27271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 14:59:54.892210 27271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 14:59:54.892390 27271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 14:59:54.892582 27271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 14:59:54.903048 27271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:41247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=45725
--tserver_master_addrs=127.11.175.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 14:59:54.904402 27271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 14:59:54.905838 27271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 14:59:54.920378 27279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:54.920468 27277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 14:59:54.920410 27276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 14:59:54.921303 27271 server_base.cc:1034] running on GCE node
I20260619 14:59:54.922042 27271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 14:59:54.923224 27271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 14:59:54.924482 27271 hybrid_clock.cc:648] HybridClock initialized: now 1781881194924427 us; error 47 us; skew 500 ppm
I20260619 14:59:54.924862 27271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 14:59:54 dist-test-slave-76rk krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881194, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 14:59:54.928094 27271 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 14:59:54.929656 27271 webserver.cc:460] Webserver started at http://127.11.175.195:45725/ using document root <none> and password file <none>
I20260619 14:59:54.930366 27271 fs_manager.cc:359] Metadata directory not provided
I20260619 14:59:54.930526 27271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 14:59:54.936097 27271 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 14:59:54.938913 27286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:54.940246 27271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 14:59:54.940438 27271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "fab9a33b44db47808f10e7e8407d63c3"
format_stamp: "Formatted at 2026-06-19 14:59:49 on dist-test-slave-76rk"
server_key: "2468343ff45c9f7af3d5403cfc2e7fb2"
server_key_iv: "daff209918535d4bb8197c0b9263b140"
server_key_version: "encryptionkey@0"
I20260619 14:59:54.941439 27271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 14:59:54.979341 27271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 14:59:54.983692 27271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 14:59:54.983991 27271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 14:59:54.985061 27271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 14:59:54.986439 27271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 14:59:54.986564 27271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:54.986720 27271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 14:59:54.986809 27271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 14:59:55.021733 27271 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:41247
I20260619 14:59:55.021858 27398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:41247 every 8 connection(s)
I20260619 14:59:55.023339 27271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 14:59:55.028623 27401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:55.026557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:44735)
0619 14:59:55.027436 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:55.027672 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":323,"threads_started":1}
W20260619 14:59:55.029417 27399 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:42475 (0 consecutive failures): Network error: Failed to ping master at 127.11.175.254:42475: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
I20260619 14:59:55.030373 27401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:55.029981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:55481)
0619 14:59:55.030153 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:55.030231 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 14:59:55.031298 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 27271
I20260619 14:59:55.031610 27401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:55.031187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:54773)
0619 14:59:55.031355 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:55.031492 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260619 14:59:55.031932 27399 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 14:59:55.084846 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:55.084047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:34481)
0619 14:59:55.084510 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:55.084594 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":213,"threads_started":1}
I20260619 14:59:55.540973 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:55.540064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:45031)
0619 14:59:55.540554 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:55.540685 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":234,"threads_started":1}
I20260619 14:59:56.033456 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:56.032560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:60883)
0619 14:59:56.033089 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:56.033222 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":293,"threads_started":1}
I20260619 14:59:56.086822 27406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:56.085929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:58945)
0619 14:59:56.086465 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:56.086570 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":310,"threads_started":1}
I20260619 14:59:56.543094 27407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:56.542022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:44305)
0619 14:59:56.542688 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:56.542858 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":313,"threads_started":1}
I20260619 14:59:57.035336 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:57.034483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:54905)
0619 14:59:57.035075 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:57.035162 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":383,"threads_started":1}
I20260619 14:59:57.088502 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:57.087752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:52167)
0619 14:59:57.088201 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:57.088297 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":230,"threads_started":1}
I20260619 14:59:57.544864 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:57.544153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:57355)
0619 14:59:57.544609 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:57.544694 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":238,"threads_started":1}
I20260619 14:59:58.037031 27411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:58.036277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:50285)
0619 14:59:58.036732 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:58.036820 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
I20260619 14:59:58.090479 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:58.089573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:42673)
0619 14:59:58.090155 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:58.090277 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":335,"threads_started":1}
I20260619 14:59:58.546715 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:58.545846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:35085)
0619 14:59:58.546361 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:58.546453 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":263,"threads_started":1}
I20260619 14:59:59.038987 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:59.038016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:53173)
0619 14:59:59.038577 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:59.038721 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":301,"threads_started":1}
I20260619 14:59:59.092263 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:59.091523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:57449)
0619 14:59:59.092028 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:59.092112 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":295,"threads_started":1}
I20260619 14:59:59.548803 27416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 14:59:59.547908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:58861)
0619 14:59:59.548461 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0619 14:59:59.548553 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":276,"threads_started":1}
I20260619 15:00:00.041285 27417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:00.040191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:53543)
0619 15:00:00.040752 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:00.040861 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":325,"threads_started":1}
I20260619 15:00:00.094571 27418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:00.093477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:38121)
0619 15:00:00.094196 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:00.094318 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":374,"threads_started":1}
I20260619 15:00:00.550707 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:00.549873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:41709)
0619 15:00:00.550367 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:00.550469 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":264,"threads_started":1}
I20260619 15:00:01.043411 27420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:01.042458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:54101)
0619 15:00:01.043149 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:01.043246 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":474,"threads_started":1}
I20260619 15:00:01.096858 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:01.095942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:52515)
0619 15:00:01.096527 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:01.096630 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":241,"threads_started":1}
I20260619 15:00:01.552567 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:01.551709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:38025)
0619 15:00:01.552232 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:01.552349 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":240,"threads_started":1}
I20260619 15:00:02.045573 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:02.044576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:37871)
0619 15:00:02.045258 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:02.045350 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":221,"threads_started":1}
I20260619 15:00:02.098837 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:02.097960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:56387)
0619 15:00:02.098488 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:02.098577 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":321,"threads_started":1}
I20260619 15:00:02.554422 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:02.553670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:45531)
0619 15:00:02.554159 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:02.554246 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":264,"threads_started":1}
I20260619 15:00:03.047646 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:03.046844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:57195)
0619 15:00:03.047357 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:03.047470 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":282,"threads_started":1}
I20260619 15:00:03.100942 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:03.100019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:54201)
0619 15:00:03.100621 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:03.100726 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":316,"threads_started":1}
I20260619 15:00:03.556268 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:03.555442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:45797)
0619 15:00:03.556007 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:03.556092 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":251,"threads_started":1}
I20260619 15:00:04.049340 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:04.048609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:43863)
0619 15:00:04.049075 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:04.049169 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":226,"threads_started":1}
I20260619 15:00:04.103541 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:04.102397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:33763)
0619 15:00:04.103233 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:04.103349 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":537,"threads_started":1}
I20260619 15:00:04.558372 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:04.557456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:60219)
0619 15:00:04.558100 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:04.558202 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":323,"threads_started":1}
I20260619 15:00:05.051177 27432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:05.050382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:45573)
0619 15:00:05.050896 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:05.051017 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":305,"threads_started":1}
I20260619 15:00:05.105271 27433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:05.104533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:45575)
0619 15:00:05.105002 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:05.105099 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":236,"threads_started":1}
I20260619 15:00:05.560151 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:05.559400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:46297)
0619 15:00:05.559852 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:05.559953 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":229,"threads_started":1}
I20260619 15:00:06.053035 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:06.052250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:45691)
0619 15:00:06.052749 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:06.052850 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":269,"threads_started":1}
I20260619 15:00:06.107009 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:06.106234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:59311)
0619 15:00:06.106738 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:06.106823 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":260,"threads_started":1}
I20260619 15:00:06.561998 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:06.561187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:59817)
0619 15:00:06.561683 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:06.561786 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":260,"threads_started":1}
I20260619 15:00:07.054924 27438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:07.054089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:37299)
0619 15:00:07.054563 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:07.054725 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":243,"threads_started":1}
I20260619 15:00:07.108916 27439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:07.108054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:55151)
0619 15:00:07.108617 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:07.108726 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":319,"threads_started":1}
I20260619 15:00:07.563861 27440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:07.563080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:55759)
0619 15:00:07.563571 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:07.563671 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":223,"threads_started":1}
I20260619 15:00:08.056857 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:08.056022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:32781)
0619 15:00:08.056546 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:08.056649 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":279,"threads_started":1}
I20260619 15:00:08.111006 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:08.110053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:50395)
0619 15:00:08.110545 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:08.110734 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":249,"threads_started":1}
I20260619 15:00:08.565735 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:08.564979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:60387)
0619 15:00:08.565460 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:08.565552 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":249,"threads_started":1}
I20260619 15:00:09.058792 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:09.057961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:60653)
0619 15:00:09.058436 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:09.058535 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":251,"threads_started":1}
I20260619 15:00:09.113067 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:09.112290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:41093)
0619 15:00:09.112783 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:09.112901 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":249,"threads_started":1}
I20260619 15:00:09.567951 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:09.567168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.194:57785)
0619 15:00:09.567625 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:09.567726 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":237,"threads_started":1}
I20260619 15:00:10.039518 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 26973
I20260619 15:00:10.061149 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:10.060081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.195:49897)
0619 15:00:10.060722 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:10.060870 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":297,"threads_started":1}
I20260619 15:00:10.070581 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:10.040914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53396 (local address 127.11.175.193:36347)
0619 15:00:10.041663 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:10.041677 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:10.041718 (+ 41us) server_negotiation.cc:407] Connection header received
0619 15:00:10.043519 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:10.043531 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:10.043991 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:10.044376 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:10.046117 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:10.047600 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:10.049047 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:10.049822 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:10.050029 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:10.062111 (+ 12082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:10.062121 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:10.062141 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:10.062197 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:10.065863 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:10.066967 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:10.066977 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:10.066986 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:10.067224 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:10.067757 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:10.067770 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:10.067772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:10.068483 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:10.068739 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:10.069622 (+ 883us) server_negotiation.cc:299] Negotiation successful
0619 15:00:10.070223 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":423,"threads_started":1}
I20260619 15:00:10.072583 27132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:10.163043 27450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:10.161816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42475 (local address 127.11.175.193:44995)
0619 15:00:10.162464 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:10.162764 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:42475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":290,"threads_started":1}
I20260619 15:00:10.163267 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 26973
I20260619 15:00:10.181591 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 27140
I20260619 15:00:10.201490 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:10.182590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.11.175.194:41057)
0619 15:00:10.183252 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:10.183266 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:10.183304 (+ 38us) server_negotiation.cc:407] Connection header received
0619 15:00:10.184905 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:10.184915 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:10.185370 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:10.185766 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:10.187591 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:10.188864 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:10.190156 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:10.190985 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:10.191221 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:10.194162 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:10.194171 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:10.194190 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:10.194238 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:10.197674 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:10.198481 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:10.198489 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:10.198496 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:10.198759 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:10.199218 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:10.199223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:10.199226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:10.199833 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:10.200197 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:10.200704 (+ 507us) server_negotiation.cc:299] Negotiation successful
0619 15:00:10.201188 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":395,"threads_started":1}
I20260619 15:00:10.203140 27263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:10.288895 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 27140
I20260619 15:00:10.307278 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 27271
I20260619 15:00:10.327028 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:10.308259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56454 (local address 127.11.175.195:41247)
0619 15:00:10.308864 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:10.308878 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:10.308921 (+ 43us) server_negotiation.cc:407] Connection header received
0619 15:00:10.310569 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:10.310580 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:10.311109 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:10.311412 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:10.312814 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:10.314261 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:10.315788 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:10.316524 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:10.316684 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:10.319948 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:10.319956 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:10.319973 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:10.320064 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:10.323375 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:10.324240 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:10.324248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:10.324256 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:10.324452 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:10.324867 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:10.324871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:10.324873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:10.325531 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:10.325796 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:10.326196 (+ 400us) server_negotiation.cc:299] Negotiation successful
0619 15:00:10.326555 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":311,"threads_started":1}
I20260619 15:00:10.328604 27394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:10.414080 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 27271
2026-06-19T15:00:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24719 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:00:10 dist-test-slave-76rk krb5kdc[27457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:00:10 dist-test-slave-76rk krb5kdc[27457](info): set up 2 sockets
Jun 19 15:00:10 dist-test-slave-76rk krb5kdc[27457](info): commencing operation
krb5kdc: starting...
W20260619 15:00:11.567420 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.004s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:00:11 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:00:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:00:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:00:11.854923 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41129
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:41129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:00:12.193922 27470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:12.194234 27470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:12.194430 27470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:12.201869 27470 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 15:00:12.201979 27470 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 15:00:12.202020 27470 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 15:00:12.202128 27470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:00:12.202205 27470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:12.202310 27470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:12.202375 27470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:00:12.202490 27470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:00:12.212805 27470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:41129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:12.214146 27470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:12.215570 27470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:12.228703 27476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:12.228892 27475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:12.228734 27478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:12.229732 27470 server_base.cc:1034] running on GCE node
I20260619 15:00:12.230590 27470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:12.232164 27470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:12.233402 27470 hybrid_clock.cc:648] HybridClock initialized: now 1781881212233339 us; error 32 us; skew 500 ppm
I20260619 15:00:12.233760 27470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:12 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:12.237121 27470 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:00:12.238787 27470 webserver.cc:460] Webserver started at http://127.11.175.254:41931/ using document root <none> and password file <none>
I20260619 15:00:12.239586 27470 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:12.239718 27470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:12.240051 27470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:00:12.243287 27470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "a29bdcdf19004ad5a67ca8410c28bb22"
format_stamp: "Formatted at 2026-06-19 15:00:12 on dist-test-slave-76rk"
server_key: "10a88df01af4fd285c1699ad7a0dd49b"
server_key_iv: "2eda5c3b1c584c844959fc826fd183bb"
server_key_version: "encryptionkey@0"
I20260619 15:00:12.244146 27470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "a29bdcdf19004ad5a67ca8410c28bb22"
format_stamp: "Formatted at 2026-06-19 15:00:12 on dist-test-slave-76rk"
server_key: "10a88df01af4fd285c1699ad7a0dd49b"
server_key_iv: "2eda5c3b1c584c844959fc826fd183bb"
server_key_version: "encryptionkey@0"
I20260619 15:00:12.249743 27470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 15:00:12.253775 27485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:12.255507 27470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 15:00:12.255736 27470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "a29bdcdf19004ad5a67ca8410c28bb22"
format_stamp: "Formatted at 2026-06-19 15:00:12 on dist-test-slave-76rk"
server_key: "10a88df01af4fd285c1699ad7a0dd49b"
server_key_iv: "2eda5c3b1c584c844959fc826fd183bb"
server_key_version: "encryptionkey@0"
I20260619 15:00:12.255999 27470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:12.285975 27470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:12.290771 27470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:12.291066 27470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:12.310767 27470 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:41129
I20260619 15:00:12.310726 27536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:41129 every 8 connection(s)
I20260619 15:00:12.312201 27470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:00:12.316160 27537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:00:12.320962 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 27470
I20260619 15:00:12.321209 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:00:12.321887 11967 external_mini_cluster.cc:1428] Setting key 3a82a7da30ded702763cb3875027feb1
I20260619 15:00:12.328753 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Bootstrap starting.
I20260619 15:00:12.332660 27537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Neither blocks nor log segments found. Creating new log.
I20260619 15:00:12.333743 27537 log.cc:824] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Log is configured to *not* fsync() on all Append() calls
I20260619 15:00:12.336817 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: No bootstrap required, opened a new log
Jun 19 15:00:12 dist-test-slave-76rk krb5kdc[27457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:00:12.346166 27537 raft_consensus.cc:348] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:12.346751 27537 raft_consensus.cc:374] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:00:12.346930 27537 raft_consensus.cc:729] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a29bdcdf19004ad5a67ca8410c28bb22, State: Initialized, Role: FOLLOWER
I20260619 15:00:12.347616 27537 consensus_queue.cc:260] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:12.347879 27537 raft_consensus.cc:388] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:00:12.348048 27537 raft_consensus.cc:482] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:00:12.348240 27537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:00:12.348354 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:12.324099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56616 (local address 127.11.175.254:41129)
0619 15:00:12.325029 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:12.325048 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:12.325109 (+ 61us) server_negotiation.cc:407] Connection header received
0619 15:00:12.327571 (+ 2462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:12.327602 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:12.328127 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:12.328617 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:12.330107 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:12.332210 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:12.333706 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:12.334791 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:12.334941 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:12.339721 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:12.339730 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:12.339749 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:12.339797 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:12.343330 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:12.344337 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:12.344345 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:12.344352 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:12.344559 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:12.345047 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:12.345052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:12.345054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:12.345695 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:12.345921 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:12.346441 (+ 520us) server_negotiation.cc:299] Negotiation successful
0619 15:00:12.347004 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":379,"threads_started":1}
I20260619 15:00:12.350767 27537 raft_consensus.cc:504] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:12.351351 27537 leader_election.cc:302] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a29bdcdf19004ad5a67ca8410c28bb22; no voters:
I20260619 15:00:12.351959 27537 leader_election.cc:288] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:00:12.352128 27542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:00:12.353737 27542 raft_consensus.cc:686] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 1 LEADER]: Becoming Leader. State: Replica: a29bdcdf19004ad5a67ca8410c28bb22, State: Running, Role: LEADER
I20260619 15:00:12.354375 27537 sys_catalog.cc:564] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:00:12.354238 27542 consensus_queue.cc:237] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:12.357364 27544 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a29bdcdf19004ad5a67ca8410c28bb22. Latest consensus state: current_term: 1 leader_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } } }
I20260619 15:00:12.357417 27543 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } } }
I20260619 15:00:12.357879 27544 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: This master's current role is: LEADER
I20260619 15:00:12.357905 27543 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: This master's current role is: LEADER
I20260619 15:00:12.360116 27550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:00:12.365881 27550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:00:12.375102 27550 catalog_manager.cc:1347] Generated new cluster ID: 3cb95f960a2f416c933708481afb4950
I20260619 15:00:12.375205 27550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:00:12.386006 27550 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:00:12.387486 27550 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:00:12.395823 27550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Generated new TSK 0
I20260619 15:00:12.396967 27550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 15:00:12.464376 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41129
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:00:12.811153 27563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:12.811476 27563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:12.811647 27563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:12.819271 27563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:12.819427 27563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:12.819617 27563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 15:00:12.829800 27563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:12.831315 27563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:12.832692 27563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:12.846289 27568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:12.846753 27571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:12.847057 27569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:12.848390 27563 server_base.cc:1034] running on GCE node
I20260619 15:00:12.848989 27563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:12.850101 27563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:12.851330 27563 hybrid_clock.cc:648] HybridClock initialized: now 1781881212851263 us; error 65 us; skew 500 ppm
I20260619 15:00:12.851682 27563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:12 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:12.855506 27563 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 15:00:12.857335 27563 webserver.cc:460] Webserver started at http://127.11.175.193:38239/ using document root <none> and password file <none>
I20260619 15:00:12.858134 27563 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:12.858279 27563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:12.858608 27563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:00:12.862295 27563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "e6cc69b514fe4fbba619375613fe8120"
format_stamp: "Formatted at 2026-06-19 15:00:12 on dist-test-slave-76rk"
server_key: "8fc0a7f2e6fe46c1cc345e6a5e20517f"
server_key_iv: "716c91d406a28762f74690744688cee4"
server_key_version: "encryptionkey@0"
I20260619 15:00:12.863574 27563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "e6cc69b514fe4fbba619375613fe8120"
format_stamp: "Formatted at 2026-06-19 15:00:12 on dist-test-slave-76rk"
server_key: "8fc0a7f2e6fe46c1cc345e6a5e20517f"
server_key_iv: "716c91d406a28762f74690744688cee4"
server_key_version: "encryptionkey@0"
I20260619 15:00:12.871444 27563 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260619 15:00:12.875430 27578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:12.877235 27563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260619 15:00:12.877463 27563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "e6cc69b514fe4fbba619375613fe8120"
format_stamp: "Formatted at 2026-06-19 15:00:12 on dist-test-slave-76rk"
server_key: "8fc0a7f2e6fe46c1cc345e6a5e20517f"
server_key_iv: "716c91d406a28762f74690744688cee4"
server_key_version: "encryptionkey@0"
I20260619 15:00:12.877715 27563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:12.898190 27563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:12.903270 27563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:12.903607 27563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:12.904719 27563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:00:12.906240 27563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:00:12.906349 27563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:12.906502 27563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:00:12.906579 27563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:12.942425 27563 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:43379
I20260619 15:00:12.942528 27690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:43379 every 8 connection(s)
I20260619 15:00:12.943864 27563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 15:00:12.950286 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 27563
I20260619 15:00:12.950564 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 15:00:12.951232 11967 external_mini_cluster.cc:1428] Setting key a5ea8dd8ccd46cebe61e7440740a7b55
Jun 19 15:00:12 dist-test-slave-76rk krb5kdc[27457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:00:12.967279 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:12.946879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:34441 (local address 127.11.175.254:41129)
0619 15:00:12.947390 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:12.947395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:12.948513 (+ 1118us) server_negotiation.cc:407] Connection header received
0619 15:00:12.950434 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:12.950440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:12.950556 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:12.950713 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:12.952613 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:12.953598 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:12.955524 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:12.955976 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:12.956129 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:12.960093 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:12.960102 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:12.960111 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:12.960194 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:12.963594 (+ 3400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:12.964676 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:12.964681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:12.964683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:12.964812 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:12.965362 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:12.965366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:12.965368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:12.965670 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:12.965786 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:12.966824 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0619 15:00:12.967012 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":275,"threads_started":1}
I20260619 15:00:12.968792 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:12.947525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:34441)
0619 15:00:12.948323 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:12.948367 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:12.950106 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:12.950969 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:12.950978 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:12.951618 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:00:12.952411 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:12.952432 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:12.953777 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:12.953781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:12.955369 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:12.955382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:12.956237 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:12.956247 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:12.956494 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:12.957252 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:00:12.957276 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:00:12.959837 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:00:12.963776 (+ 3939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:12.963790 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:12.963812 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:12.964514 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:12.964930 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:12.964938 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:12.964945 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:12.965207 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:12.965818 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:00:12.965829 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:00:12.966305 (+ 476us) client_negotiation.cc:769] Sending connection context
0619 15:00:12.966629 (+ 324us) client_negotiation.cc:240] Negotiation successful
0619 15:00:12.967289 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":289,"threads_started":1}
I20260619 15:00:12.970613 27691 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41129
I20260619 15:00:12.971027 27691 heartbeater.cc:464] Registering TS with master...
I20260619 15:00:12.971972 27691 heartbeater.cc:511] Master 127.11.175.254:41129 requested a full tablet report, sending...
I20260619 15:00:12.974334 27502 ts_manager.cc:194] Registered new tserver with Master: e6cc69b514fe4fbba619375613fe8120 (127.11.175.193:43379)
I20260619 15:00:12.975912 27502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:34441
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 15:00:13.014446 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41129
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:00:13.361323 27697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:13.361630 27697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:13.361752 27697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:13.369469 27697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:13.369604 27697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:13.369818 27697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 15:00:13.380457 27697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:13.381800 27697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:13.383293 27697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:13.397285 27703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:13.397220 27702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:13.397330 27705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:13.398219 27697 server_base.cc:1034] running on GCE node
I20260619 15:00:13.398929 27697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:13.399891 27697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:13.401095 27697 hybrid_clock.cc:648] HybridClock initialized: now 1781881213401046 us; error 38 us; skew 500 ppm
I20260619 15:00:13.401435 27697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:13 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881213, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:13.404834 27697 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 15:00:13.406507 27697 webserver.cc:460] Webserver started at http://127.11.175.194:45801/ using document root <none> and password file <none>
I20260619 15:00:13.407286 27697 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:13.407442 27697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:13.407768 27697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:00:13.411100 27697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "decc6cea2acd4f1c850d066d19a55154"
format_stamp: "Formatted at 2026-06-19 15:00:13 on dist-test-slave-76rk"
server_key: "b124dec6758fb893dc0fe1c2016eafb1"
server_key_iv: "5b824b84b028e1efd7ff2d1ceaace621"
server_key_version: "encryptionkey@0"
I20260619 15:00:13.411952 27697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "decc6cea2acd4f1c850d066d19a55154"
format_stamp: "Formatted at 2026-06-19 15:00:13 on dist-test-slave-76rk"
server_key: "b124dec6758fb893dc0fe1c2016eafb1"
server_key_iv: "5b824b84b028e1efd7ff2d1ceaace621"
server_key_version: "encryptionkey@0"
I20260619 15:00:13.417512 27697 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 15:00:13.421461 27712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:13.423050 27697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 15:00:13.423259 27697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "decc6cea2acd4f1c850d066d19a55154"
format_stamp: "Formatted at 2026-06-19 15:00:13 on dist-test-slave-76rk"
server_key: "b124dec6758fb893dc0fe1c2016eafb1"
server_key_iv: "5b824b84b028e1efd7ff2d1ceaace621"
server_key_version: "encryptionkey@0"
I20260619 15:00:13.423493 27697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:13.444049 27697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:13.449468 27697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:13.449867 27697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:13.451097 27697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:00:13.452577 27697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:00:13.452685 27697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:13.452839 27697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:00:13.452912 27697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:13.489395 27697 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:45713
I20260619 15:00:13.489437 27824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:45713 every 8 connection(s)
I20260619 15:00:13.490813 27697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 15:00:13.501029 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 27697
I20260619 15:00:13.501233 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 15:00:13.501710 11967 external_mini_cluster.cc:1428] Setting key 9b0ef4ec5fa592b9f625cbe82b44859b
Jun 19 15:00:13 dist-test-slave-76rk krb5kdc[27457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881213, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:00:13.512115 27827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:13.493644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:57623 (local address 127.11.175.254:41129)
0619 15:00:13.494099 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:13.494104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:13.495328 (+ 1224us) server_negotiation.cc:407] Connection header received
0619 15:00:13.497239 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:13.497244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:13.497338 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:13.497462 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:13.499116 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:13.499826 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:13.501502 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:13.501984 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:13.502119 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:13.505835 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:13.505841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:13.505844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:13.505880 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:13.508813 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:13.509621 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:13.509631 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:13.509633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:13.509769 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:13.510242 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:13.510251 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:13.510253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:13.510547 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:13.510749 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:13.511481 (+ 732us) server_negotiation.cc:299] Negotiation successful
0619 15:00:13.511719 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":232,"threads_started":1}
I20260619 15:00:13.513161 27828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:13.494293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:57623)
0619 15:00:13.495148 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:13.495193 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:13.496960 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:13.497706 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:13.497714 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:13.498236 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:00:13.498940 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:13.498959 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:13.499987 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:13.499993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:13.501359 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:13.501372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:13.502225 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:13.502232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:13.502476 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:13.503298 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:00:13.503326 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:00:13.505575 (+ 2249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:00:13.508974 (+ 3399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:13.508984 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:13.509004 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:13.509469 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:13.509891 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:13.509896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:13.509900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:13.510115 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:13.510709 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:00:13.510717 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:00:13.511101 (+ 384us) client_negotiation.cc:769] Sending connection context
0619 15:00:13.511379 (+ 278us) client_negotiation.cc:240] Negotiation successful
0619 15:00:13.511971 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":380,"threads_started":1}
I20260619 15:00:13.514838 27825 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41129
I20260619 15:00:13.515182 27825 heartbeater.cc:464] Registering TS with master...
I20260619 15:00:13.516054 27825 heartbeater.cc:511] Master 127.11.175.254:41129 requested a full tablet report, sending...
I20260619 15:00:13.517727 27502 ts_manager.cc:194] Registered new tserver with Master: decc6cea2acd4f1c850d066d19a55154 (127.11.175.194:45713)
I20260619 15:00:13.519032 27502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:57623
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 15:00:13.558749 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41129
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:00:13.899822 27831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:13.900130 27831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:13.900225 27831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:13.907703 27831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:13.907833 27831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:13.907955 27831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 15:00:13.919867 27831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:41129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:13.921279 27831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:13.922623 27831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:13.936430 27839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:13.936439 27836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:13.936448 27837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:13.937309 27831 server_base.cc:1034] running on GCE node
I20260619 15:00:13.938022 27831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:13.939098 27831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:13.940307 27831 hybrid_clock.cc:648] HybridClock initialized: now 1781881213940255 us; error 45 us; skew 500 ppm
I20260619 15:00:13.940649 27831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:13 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881213, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:13.944667 27831 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 15:00:13.948736 27831 webserver.cc:460] Webserver started at http://127.11.175.195:35889/ using document root <none> and password file <none>
I20260619 15:00:13.949445 27831 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:13.949591 27831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:13.949949 27831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:00:13.952970 27831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de"
format_stamp: "Formatted at 2026-06-19 15:00:13 on dist-test-slave-76rk"
server_key: "cf3143ed25236ef2c5882aad3dc39237"
server_key_iv: "79b64421c6e6efeea25f9e54a127b200"
server_key_version: "encryptionkey@0"
I20260619 15:00:13.953805 27831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de"
format_stamp: "Formatted at 2026-06-19 15:00:13 on dist-test-slave-76rk"
server_key: "cf3143ed25236ef2c5882aad3dc39237"
server_key_iv: "79b64421c6e6efeea25f9e54a127b200"
server_key_version: "encryptionkey@0"
I20260619 15:00:13.959924 27831 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260619 15:00:13.964130 27846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:13.965629 27831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 15:00:13.965838 27831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de"
format_stamp: "Formatted at 2026-06-19 15:00:13 on dist-test-slave-76rk"
server_key: "cf3143ed25236ef2c5882aad3dc39237"
server_key_iv: "79b64421c6e6efeea25f9e54a127b200"
server_key_version: "encryptionkey@0"
I20260619 15:00:13.966084 27831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:13.979848 27691 heartbeater.cc:503] Master 127.11.175.254:41129 was elected leader, sending a full tablet report...
I20260619 15:00:13.985527 27831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:13.990321 27831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:13.990592 27831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:13.991686 27831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:00:13.993218 27831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:00:13.993292 27831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:13.993391 27831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:00:13.993436 27831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:14.028312 27831 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:36139
I20260619 15:00:14.028409 27958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:36139 every 8 connection(s)
I20260619 15:00:14.029660 27831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 15:00:14.035517 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 27831
I20260619 15:00:14.035744 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 15:00:14.036424 11967 external_mini_cluster.cc:1428] Setting key e51b69c70f0944d8efa2008717e9b81d
Jun 19 15:00:14 dist-test-slave-76rk krb5kdc[27457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881213, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:00:14.050781 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.032393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:58681 (local address 127.11.175.254:41129)
0619 15:00:14.032907 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.032913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.033896 (+ 983us) server_negotiation.cc:407] Connection header received
0619 15:00:14.035917 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.035923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.036053 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.036182 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:14.037749 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.038586 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.040212 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.040616 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.040751 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.044386 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:14.044392 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:14.044395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:14.044438 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:14.047359 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:14.048153 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:14.048159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:14.048163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:14.048299 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:14.048733 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:14.048749 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:14.048753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:14.049084 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:14.049255 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.050071 (+ 816us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.050312 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":232,"threads_started":1}
I20260619 15:00:14.051889 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.032995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:58681)
0619 15:00:14.033712 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:14.033770 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:14.035565 (+ 1795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:14.036379 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:14.036391 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:14.037015 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:00:14.037578 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.037595 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.038777 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.038781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.040054 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.040067 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.040848 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.040858 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.041108 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.041762 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:00:14.041801 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:00:14.044150 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:00:14.047530 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:14.047539 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:14.047558 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:14.048019 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:14.048412 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:14.048417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:14.048421 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:14.048617 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:14.049219 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:00:14.049227 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:00:14.049628 (+ 401us) client_negotiation.cc:769] Sending connection context
0619 15:00:14.049958 (+ 330us) client_negotiation.cc:240] Negotiation successful
0619 15:00:14.050536 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":281,"threads_started":1}
I20260619 15:00:14.053306 27959 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41129
I20260619 15:00:14.053583 27959 heartbeater.cc:464] Registering TS with master...
I20260619 15:00:14.054311 27959 heartbeater.cc:511] Master 127.11.175.254:41129 requested a full tablet report, sending...
I20260619 15:00:14.056157 27502 ts_manager.cc:194] Registered new tserver with Master: 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de (127.11.175.195:36139)
I20260619 15:00:14.057125 27502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:58681
I20260619 15:00:14.065058 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 15:00:14.084692 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.069865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56628 (local address 127.11.175.254:41129)
0619 15:00:14.070156 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.070168 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.070574 (+ 406us) server_negotiation.cc:407] Connection header received
0619 15:00:14.070763 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.070768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.070839 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.070975 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:14.072595 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.073367 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.074952 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.075592 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.075717 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.078346 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:14.078354 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:14.078358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:14.078404 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:14.081529 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:14.082546 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:14.082551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:14.082553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:14.082783 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:14.083222 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:14.083227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:14.083229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:14.083597 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:14.083810 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.084237 (+ 427us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.084441 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260619 15:00:14.089736 27502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 15:00:14.096691 27502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 15:00:14.133176 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.110265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41888 (local address 127.11.175.194:45713)
0619 15:00:14.124332 (+ 14067us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.124344 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.124383 (+ 39us) server_negotiation.cc:407] Connection header received
0619 15:00:14.124515 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.124522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.124834 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.125074 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:00:14.126027 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.127563 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.130409 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.131985 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.132154 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.132350 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.132654 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.132838 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13913,"thread_start_us":13775,"threads_started":1}
I20260619 15:00:14.133258 27974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.110282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:45713 (local address 127.0.0.1:41888)
0619 15:00:14.123398 (+ 13116us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:14.123422 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:14.123567 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:14.125410 (+ 1843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:14.125417 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:14.125451 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:00:14.125836 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.125850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.127727 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.127733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.130208 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.130224 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.132183 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.132189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.132392 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.132449 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 15:00:14.132595 (+ 146us) client_negotiation.cc:240] Negotiation successful
0619 15:00:14.132788 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12943,"mutex_wait_us":84,"thread_start_us":13329,"threads_started":1}
I20260619 15:00:14.134271 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.110280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:36139 (local address 127.0.0.1:49696)
0619 15:00:14.122278 (+ 11998us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:14.122359 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:14.122674 (+ 315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:14.127027 (+ 4353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:14.127031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:14.127061 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:00:14.127486 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.127503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.129304 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.129314 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.131369 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.131382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.133505 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.133510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.133620 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.133642 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 15:00:14.133751 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 15:00:14.133889 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11594,"thread_start_us":11361,"threads_started":1}
I20260619 15:00:14.134346 27975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.110564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49696 (local address 127.11.175.195:36139)
0619 15:00:14.126077 (+ 15513us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.126083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.126109 (+ 26us) server_negotiation.cc:407] Connection header received
0619 15:00:14.126183 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.126188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.126437 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.126676 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:00:14.127697 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.129161 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.131576 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.133003 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.133185 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.133361 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.133827 (+ 466us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.133994 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15342,"thread_start_us":15134,"threads_started":1}
I20260619 15:00:14.134701 27976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.113257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.0.0.1:43102)
0619 15:00:14.124857 (+ 11600us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:14.124877 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:14.125018 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:14.126307 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:14.126312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:14.126337 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:00:14.126811 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.126824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.128922 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.128928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.131546 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.131561 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.133832 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.133840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.134070 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.134111 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 15:00:14.134204 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 15:00:14.134330 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11467,"mutex_wait_us":2,"thread_start_us":11326,"threads_started":1}
I20260619 15:00:14.135290 27977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.113616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43102 (local address 127.11.175.193:43379)
0619 15:00:14.125635 (+ 12019us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.125644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.125674 (+ 30us) server_negotiation.cc:407] Connection header received
0619 15:00:14.125781 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.125792 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.126084 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.126320 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:00:14.126988 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.128778 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.131748 (+ 2970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.133309 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.133511 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.133670 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.134786 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.134979 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11822,"thread_start_us":11646,"threads_started":1}
I20260619 15:00:14.138108 27760 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0989aa96c147b191b15f71a3b5cdd7 (DEFAULT_TABLE table=test-table [id=a9ce4b5fb5d54208992a7469903aae52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:00:14.138444 27626 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0989aa96c147b191b15f71a3b5cdd7 (DEFAULT_TABLE table=test-table [id=a9ce4b5fb5d54208992a7469903aae52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:00:14.139716 27626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0989aa96c147b191b15f71a3b5cdd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:00:14.139755 27760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0989aa96c147b191b15f71a3b5cdd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:00:14.139268 27894 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0989aa96c147b191b15f71a3b5cdd7 (DEFAULT_TABLE table=test-table [id=a9ce4b5fb5d54208992a7469903aae52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:00:14.140672 27894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0989aa96c147b191b15f71a3b5cdd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:00:14.151683 27978 tablet_bootstrap.cc:492] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: Bootstrap starting.
I20260619 15:00:14.155668 27978 tablet_bootstrap.cc:654] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: Neither blocks nor log segments found. Creating new log.
I20260619 15:00:14.156490 27980 tablet_bootstrap.cc:492] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Bootstrap starting.
I20260619 15:00:14.157390 27978 log.cc:824] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: Log is configured to *not* fsync() on all Append() calls
I20260619 15:00:14.158522 27979 tablet_bootstrap.cc:492] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: Bootstrap starting.
I20260619 15:00:14.160648 27980 tablet_bootstrap.cc:654] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Neither blocks nor log segments found. Creating new log.
I20260619 15:00:14.161988 27980 log.cc:824] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Log is configured to *not* fsync() on all Append() calls
I20260619 15:00:14.162041 27979 tablet_bootstrap.cc:654] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: Neither blocks nor log segments found. Creating new log.
I20260619 15:00:14.162277 27978 tablet_bootstrap.cc:492] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: No bootstrap required, opened a new log
I20260619 15:00:14.162617 27978 ts_tablet_manager.cc:1397] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260619 15:00:14.163393 27979 log.cc:824] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: Log is configured to *not* fsync() on all Append() calls
I20260619 15:00:14.165755 27980 tablet_bootstrap.cc:492] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: No bootstrap required, opened a new log
I20260619 15:00:14.166117 27980 ts_tablet_manager.cc:1397] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260619 15:00:14.169734 27979 tablet_bootstrap.cc:492] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: No bootstrap required, opened a new log
I20260619 15:00:14.170080 27979 ts_tablet_manager.cc:1397] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260619 15:00:14.171764 27978 raft_consensus.cc:348] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } }
I20260619 15:00:14.172178 27978 raft_consensus.cc:374] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:00:14.172302 27978 raft_consensus.cc:729] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6cc69b514fe4fbba619375613fe8120, State: Initialized, Role: FOLLOWER
I20260619 15:00:14.172953 27978 consensus_queue.cc:260] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } }
I20260619 15:00:14.176247 27978 ts_tablet_manager.cc:1428] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: Time spent starting tablet: real 0.013s user 0.008s sys 0.003s
I20260619 15:00:14.177232 27980 raft_consensus.cc:348] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } }
I20260619 15:00:14.177740 27980 raft_consensus.cc:374] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:00:14.177868 27980 raft_consensus.cc:729] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de, State: Initialized, Role: FOLLOWER
I20260619 15:00:14.178583 27980 consensus_queue.cc:260] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } }
I20260619 15:00:14.180508 27980 ts_tablet_manager.cc:1428] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260619 15:00:14.180565 27959 heartbeater.cc:503] Master 127.11.175.254:41129 was elected leader, sending a full tablet report...
I20260619 15:00:14.182106 27979 raft_consensus.cc:348] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } }
I20260619 15:00:14.182665 27979 raft_consensus.cc:374] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:00:14.182780 27979 raft_consensus.cc:729] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: decc6cea2acd4f1c850d066d19a55154, State: Initialized, Role: FOLLOWER
I20260619 15:00:14.183421 27979 consensus_queue.cc:260] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } }
I20260619 15:00:14.184722 27825 heartbeater.cc:503] Master 127.11.175.254:41129 was elected leader, sending a full tablet report...
I20260619 15:00:14.185359 27979 ts_tablet_manager.cc:1428] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260619 15:00:14.196648 27692 tablet.cc:2369] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 15:00:14.243044 27826 tablet.cc:2369] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 15:00:14.281327 27960 tablet.cc:2369] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:00:14.420647 27984 raft_consensus.cc:482] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 15:00:14.421000 27984 raft_consensus.cc:504] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } }
I20260619 15:00:14.422789 27984 leader_election.cc:288] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers decc6cea2acd4f1c850d066d19a55154 (127.11.175.194:45713), 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de (127.11.175.195:36139)
I20260619 15:00:14.429556 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.423201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:55603 (local address 127.11.175.194:45713)
0619 15:00:14.423460 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.423466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.423880 (+ 414us) server_negotiation.cc:407] Connection header received
0619 15:00:14.424199 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.424204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.424266 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.424364 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:00:14.424928 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.425650 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.427684 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.428507 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.428688 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.428767 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.429128 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.429279 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 15:00:14.429641 27987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.422984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:45713 (local address 127.11.175.193:55603)
0619 15:00:14.423754 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:14.423778 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:14.424033 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:14.424381 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:14.424386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:14.424409 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:00:14.424778 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.424789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.425784 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.425790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.427514 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.427527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.428712 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.428718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.428960 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.428993 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 15:00:14.429077 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 15:00:14.429195 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":60,"thread_start_us":408,"threads_started":1}
I20260619 15:00:14.429612 27975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.423700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:56375 (local address 127.11.175.195:36139)
0619 15:00:14.423926 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.423932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.424073 (+ 141us) server_negotiation.cc:407] Connection header received
0619 15:00:14.424281 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.424285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.424342 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.424453 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:00:14.425115 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.425848 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.427798 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.428646 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.428830 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.428916 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.429238 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.429380 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 15:00:14.429986 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.423193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:36139 (local address 127.11.175.193:56375)
0619 15:00:14.423972 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:14.423994 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:14.424145 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:14.424473 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:14.424478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:14.424519 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:00:14.424965 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.424973 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.425993 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.425998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.427612 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:14.427624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.428861 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.428867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:14.429065 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.429102 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 15:00:14.429197 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 15:00:14.429312 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":19,"thread_start_us":566,"threads_started":1}
I20260619 15:00:14.430727 27780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0989aa96c147b191b15f71a3b5cdd7" candidate_uuid: "e6cc69b514fe4fbba619375613fe8120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "decc6cea2acd4f1c850d066d19a55154" is_pre_election: true
I20260619 15:00:14.431327 27780 raft_consensus.cc:2454] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6cc69b514fe4fbba619375613fe8120 in term 0.
I20260619 15:00:14.431721 27914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0989aa96c147b191b15f71a3b5cdd7" candidate_uuid: "e6cc69b514fe4fbba619375613fe8120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" is_pre_election: true
I20260619 15:00:14.431978 27582 leader_election.cc:302] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: decc6cea2acd4f1c850d066d19a55154, e6cc69b514fe4fbba619375613fe8120; no voters:
I20260619 15:00:14.432297 27914 raft_consensus.cc:2454] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6cc69b514fe4fbba619375613fe8120 in term 0.
I20260619 15:00:14.432399 27984 raft_consensus.cc:2781] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 15:00:14.432544 27984 raft_consensus.cc:482] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 15:00:14.432657 27984 raft_consensus.cc:3037] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:00:14.434929 27984 raft_consensus.cc:504] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } }
I20260619 15:00:14.435603 27984 leader_election.cc:288] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [CANDIDATE]: Term 1 election: Requested vote from peers decc6cea2acd4f1c850d066d19a55154 (127.11.175.194:45713), 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de (127.11.175.195:36139)
I20260619 15:00:14.436064 27780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0989aa96c147b191b15f71a3b5cdd7" candidate_uuid: "e6cc69b514fe4fbba619375613fe8120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "decc6cea2acd4f1c850d066d19a55154"
I20260619 15:00:14.436265 27780 raft_consensus.cc:3037] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:00:14.436195 27914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0989aa96c147b191b15f71a3b5cdd7" candidate_uuid: "e6cc69b514fe4fbba619375613fe8120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de"
I20260619 15:00:14.436450 27914 raft_consensus.cc:3037] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:00:14.438462 27780 raft_consensus.cc:2454] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6cc69b514fe4fbba619375613fe8120 in term 1.
I20260619 15:00:14.438848 27582 leader_election.cc:302] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: decc6cea2acd4f1c850d066d19a55154, e6cc69b514fe4fbba619375613fe8120; no voters:
I20260619 15:00:14.438998 27914 raft_consensus.cc:2454] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6cc69b514fe4fbba619375613fe8120 in term 1.
I20260619 15:00:14.439127 27984 raft_consensus.cc:2781] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:00:14.439610 27984 raft_consensus.cc:686] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 1 LEADER]: Becoming Leader. State: Replica: e6cc69b514fe4fbba619375613fe8120, State: Running, Role: LEADER
I20260619 15:00:14.440083 27984 consensus_queue.cc:237] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } }
I20260619 15:00:14.445048 27502 catalog_manager.cc:5495] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6cc69b514fe4fbba619375613fe8120 (127.11.175.193). New cstate: current_term: 1 leader_uuid: "e6cc69b514fe4fbba619375613fe8120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } health_report { overall_health: HEALTHY } } }
I20260619 15:00:14.504402 27977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.498474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43104 (local address 127.11.175.193:43379)
0619 15:00:14.498699 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.498705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.498726 (+ 21us) server_negotiation.cc:407] Connection header received
0619 15:00:14.498896 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.498902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.499000 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.499147 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:00:14.499507 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.500600 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.502139 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.502571 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.502728 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.503288 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:00:14.503792 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:00:14.503909 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.504036 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.504106 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 15:00:14.512914 27914 raft_consensus.cc:1261] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 1 FOLLOWER]: Refusing update from remote peer e6cc69b514fe4fbba619375613fe8120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:00:14.512953 27780 raft_consensus.cc:1261] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154 [term 1 FOLLOWER]: Refusing update from remote peer e6cc69b514fe4fbba619375613fe8120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:00:14.513911 27984 consensus_queue.cc:1035] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:00:14.514214 27989 consensus_queue.cc:1035] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:00:14.526335 27992 mvcc.cc:205] Tried to move back new op lower bound from 7298585454633644032 to 7298585454350073856. Current Snapshot: MvccSnapshot[applied={T|T < 7298585454633644032}]
I20260619 15:00:14.539188 27502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:00:14.539487 27502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:00:14.543339 27502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Sending DeleteTablet for 3 replicas of tablet 6a0989aa96c147b191b15f71a3b5cdd7
I20260619 15:00:14.544610 27894 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0989aa96c147b191b15f71a3b5cdd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:00:14 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:49696
I20260619 15:00:14.544626 27760 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0989aa96c147b191b15f71a3b5cdd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:00:14 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:41888
I20260619 15:00:14.544637 27626 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0989aa96c147b191b15f71a3b5cdd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:00:14 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:43102
I20260619 15:00:14.549618 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 27563
I20260619 15:00:14.557510 28003 tablet_replica.cc:330] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: stopping tablet replica
I20260619 15:00:14.557869 28002 tablet_replica.cc:330] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: stopping tablet replica
I20260619 15:00:14.558275 28002 raft_consensus.cc:2229] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:00:14.558267 28003 raft_consensus.cc:2229] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:00:14.558871 28002 raft_consensus.cc:2258] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:00:14.558871 28003 raft_consensus.cc:2258] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:00:14.561234 28003 ts_tablet_manager.cc:1905] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:00:14.561275 28002 ts_tablet_manager.cc:1905] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:00:14.566854 28004 tablet_replica.cc:330] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: stopping tablet replica
I20260619 15:00:14.567559 28004 raft_consensus.cc:2229] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 1 LEADER]: Raft consensus shutting down.
I20260619 15:00:14.568413 28004 raft_consensus.cc:2258] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:00:14.568565 28003 ts_tablet_manager.cc:1918] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:00:14.568742 28003 log.cc:1196] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/6a0989aa96c147b191b15f71a3b5cdd7
I20260619 15:00:14.569293 28003 ts_tablet_manager.cc:1939] T 6a0989aa96c147b191b15f71a3b5cdd7 P decc6cea2acd4f1c850d066d19a55154: Deleting consensus metadata
I20260619 15:00:14.571877 27489 catalog_manager.cc:4841] TS decc6cea2acd4f1c850d066d19a55154 (127.11.175.194:45713): tablet 6a0989aa96c147b191b15f71a3b5cdd7 (table test-table [id=a9ce4b5fb5d54208992a7469903aae52]) successfully deleted
I20260619 15:00:14.573027 28002 ts_tablet_manager.cc:1918] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:00:14.573218 28002 log.cc:1196] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/6a0989aa96c147b191b15f71a3b5cdd7
I20260619 15:00:14.573760 28002 ts_tablet_manager.cc:1939] T 6a0989aa96c147b191b15f71a3b5cdd7 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Deleting consensus metadata
I20260619 15:00:14.573911 28004 ts_tablet_manager.cc:1905] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:00:14.575708 27488 catalog_manager.cc:4841] TS 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de (127.11.175.195:36139): tablet 6a0989aa96c147b191b15f71a3b5cdd7 (table test-table [id=a9ce4b5fb5d54208992a7469903aae52]) successfully deleted
Jun 19 15:00:14 dist-test-slave-76rk krb5kdc[27457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 15:00:14.581350 28004 ts_tablet_manager.cc:1918] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:00:14.581540 28004 log.cc:1196] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/6a0989aa96c147b191b15f71a3b5cdd7
I20260619 15:00:14.582144 28004 ts_tablet_manager.cc:1939] T 6a0989aa96c147b191b15f71a3b5cdd7 P e6cc69b514fe4fbba619375613fe8120: Deleting consensus metadata
I20260619 15:00:14.583890 27486 catalog_manager.cc:4841] TS e6cc69b514fe4fbba619375613fe8120 (127.11.175.193:43379): tablet 6a0989aa96c147b191b15f71a3b5cdd7 (table test-table [id=a9ce4b5fb5d54208992a7469903aae52]) successfully deleted
I20260619 15:00:14.585896 27977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.550433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43106 (local address 127.11.175.193:43379)
0619 15:00:14.550683 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.550693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.555246 (+ 4553us) server_negotiation.cc:407] Connection header received
0619 15:00:14.558545 (+ 3299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.558555 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.558719 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.558818 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:14.566819 (+ 8001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.568144 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.570538 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.571340 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.571455 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.577763 (+ 6308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:14.577779 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:14.577803 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:14.577861 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:14.582402 (+ 4541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:14.583499 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:14.583511 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:14.583516 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:14.583710 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:14.584232 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:14.584236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:14.584238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:14.584705 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:14.585036 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.585369 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.585564 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 15:00:14.586655 27686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:14.687100 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 27563
I20260619 15:00:14.705473 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 27697
Jun 19 15:00:14 dist-test-slave-76rk krb5kdc[27457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 15:00:14.721732 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.706364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41904 (local address 127.11.175.194:45713)
0619 15:00:14.706561 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.706566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.706585 (+ 19us) server_negotiation.cc:407] Connection header received
0619 15:00:14.706838 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.706843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.706908 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.706995 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:14.708467 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.709199 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.710499 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.711039 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.711139 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.715927 (+ 4788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:14.715944 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:14.715961 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:14.716000 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:14.719113 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:14.719813 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:14.719820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:14.719822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:14.719949 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:14.720281 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:14.720284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:14.720286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:14.720672 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:14.720891 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.721192 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.721363 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 15:00:14.722405 27820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:14.814373 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 27697
I20260619 15:00:14.832044 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 27831
Jun 19 15:00:14 dist-test-slave-76rk krb5kdc[27457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 15:00:14.848167 27975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:14.832833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49708 (local address 127.11.175.195:36139)
0619 15:00:14.833061 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:14.833066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:14.833085 (+ 19us) server_negotiation.cc:407] Connection header received
0619 15:00:14.833215 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:14.833219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:14.833288 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:14.833373 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:14.834738 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.835493 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.836845 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:14.837249 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:14.837362 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:14.842110 (+ 4748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:14.842127 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:14.842143 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:14.842189 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:14.845126 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:14.845882 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:14.845889 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:14.845892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:14.846040 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:14.846493 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:14.846497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:14.846499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:14.846989 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:14.847231 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:14.847589 (+ 358us) server_negotiation.cc:299] Negotiation successful
0619 15:00:14.847766 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260619 15:00:14.848795 27954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:14.944552 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 27831
I20260619 15:00:14.962142 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 27470
I20260619 15:00:14.963383 27532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:15.076633 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 27470
I20260619 15:00:35.099915 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41129
--webserver_interface=127.11.175.254
--webserver_port=41931
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:41129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:00:35.464116 28010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:35.464439 28010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:35.464578 28010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:35.472976 28010 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 15:00:35.473112 28010 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 15:00:35.473192 28010 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 15:00:35.473274 28010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:00:35.473395 28010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:35.473481 28010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:35.473588 28010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:00:35.473675 28010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:00:35.484987 28010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:41129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=41931
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:35.486407 28010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:35.487865 28010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:35.515173 28019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:35.515271 28016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:35.515175 28017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:35.515470 28010 server_base.cc:1034] running on GCE node
I20260619 15:00:35.516956 28010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:35.518963 28010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:35.520267 28010 hybrid_clock.cc:648] HybridClock initialized: now 1781881235520185 us; error 58 us; skew 500 ppm
I20260619 15:00:35.520713 28010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:35 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881235, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:35.524943 28010 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:00:35.527297 28010 webserver.cc:460] Webserver started at http://127.11.175.254:41931/ using document root <none> and password file <none>
I20260619 15:00:35.528192 28010 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:35.528337 28010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:35.535336 28010 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:00:35.539158 28026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:35.540822 28010 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 15:00:35.541061 28010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "a29bdcdf19004ad5a67ca8410c28bb22"
format_stamp: "Formatted at 2026-06-19 15:00:12 on dist-test-slave-76rk"
server_key: "10a88df01af4fd285c1699ad7a0dd49b"
server_key_iv: "2eda5c3b1c584c844959fc826fd183bb"
server_key_version: "encryptionkey@0"
I20260619 15:00:35.542315 28010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:35.564129 28010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:35.569952 28010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:35.570322 28010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:35.591926 28010 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:41129
I20260619 15:00:35.591969 28077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:41129 every 8 connection(s)
I20260619 15:00:35.593706 28010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:00:35.599306 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 28010
I20260619 15:00:35.600209 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:43379
--local_ip_for_outbound_sockets=127.11.175.193
--tserver_master_addrs=127.11.175.254:41129
--webserver_port=38239
--webserver_interface=127.11.175.193
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 15:00:35.602209 28078 sys_catalog.cc:263] Verifying existing consensus state
I20260619 15:00:35.605818 28078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Bootstrap starting.
I20260619 15:00:35.626948 28078 log.cc:824] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Log is configured to *not* fsync() on all Append() calls
I20260619 15:00:35.641513 28078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 15:00:35.642740 28078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Bootstrap complete.
I20260619 15:00:35.654891 28078 raft_consensus.cc:348] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:35.655884 28078 raft_consensus.cc:729] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a29bdcdf19004ad5a67ca8410c28bb22, State: Initialized, Role: FOLLOWER
I20260619 15:00:35.656684 28078 consensus_queue.cc:260] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:35.656991 28078 raft_consensus.cc:388] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:00:35.657119 28078 raft_consensus.cc:482] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:00:35.657310 28078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 1 FOLLOWER]: Advancing to term 2
I20260619 15:00:35.662433 28078 raft_consensus.cc:504] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:35.663486 28078 leader_election.cc:302] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a29bdcdf19004ad5a67ca8410c28bb22; no voters:
I20260619 15:00:35.664345 28078 leader_election.cc:288] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 15:00:35.664510 28082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 15:00:35.667501 28078 sys_catalog.cc:564] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:00:35.671774 28082 raft_consensus.cc:686] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 2 LEADER]: Becoming Leader. State: Replica: a29bdcdf19004ad5a67ca8410c28bb22, State: Running, Role: LEADER
I20260619 15:00:35.672564 28082 consensus_queue.cc:237] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:35.676168 28083 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } } }
I20260619 15:00:35.676761 28083 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: This master's current role is: LEADER
I20260619 15:00:35.677639 28084 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a29bdcdf19004ad5a67ca8410c28bb22. Latest consensus state: current_term: 2 leader_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } } }
I20260619 15:00:35.678042 28084 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: This master's current role is: LEADER
I20260619 15:00:35.680270 28094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:00:35.692135 28094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:00:35.693998 28094 catalog_manager.cc:1259] Loaded cluster ID: 3cb95f960a2f416c933708481afb4950
I20260619 15:00:35.694228 28094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:00:35.699891 28094 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:00:35.701790 28094 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Loaded TSK: 0
I20260619 15:00:35.739241 28094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Generated new TSK 1
I20260619 15:00:35.739706 28094 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 15:00:36.006970 28080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:36.007290 28080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:36.007452 28080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:36.016024 28080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:36.016186 28080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:36.016400 28080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 15:00:36.027853 28080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:43379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=38239
--tserver_master_addrs=127.11.175.254:41129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:36.029376 28080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:36.030992 28080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:36.047479 28107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:36.047585 28106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:36.048604 28109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:36.048898 28080 server_base.cc:1034] running on GCE node
I20260619 15:00:36.049716 28080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:36.050979 28080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:36.052268 28080 hybrid_clock.cc:648] HybridClock initialized: now 1781881236052220 us; error 42 us; skew 500 ppm
I20260619 15:00:36.052683 28080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:36 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881236, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:36.056361 28080 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 15:00:36.058799 28080 webserver.cc:460] Webserver started at http://127.11.175.193:38239/ using document root <none> and password file <none>
I20260619 15:00:36.059630 28080 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:36.059806 28080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:36.066368 28080 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260619 15:00:36.069979 28116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:36.071890 28080 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:00:36.072187 28080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "e6cc69b514fe4fbba619375613fe8120"
format_stamp: "Formatted at 2026-06-19 15:00:12 on dist-test-slave-76rk"
server_key: "8fc0a7f2e6fe46c1cc345e6a5e20517f"
server_key_iv: "716c91d406a28762f74690744688cee4"
server_key_version: "encryptionkey@0"
I20260619 15:00:36.073349 28080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:36.089030 28080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:36.093909 28080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:36.094231 28080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:36.095391 28080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:00:36.096901 28080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:00:36.097039 28080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:36.097234 28080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:00:36.097308 28080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:36.134018 28080 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:43379
I20260619 15:00:36.134116 28228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:43379 every 8 connection(s)
I20260619 15:00:36.135610 28080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 15:00:36.139448 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 28080
I20260619 15:00:36.140312 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:45713
--local_ip_for_outbound_sockets=127.11.175.194
--tserver_master_addrs=127.11.175.254:41129
--webserver_port=45801
--webserver_interface=127.11.175.194
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 15:00:36 dist-test-slave-76rk krb5kdc[27457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881236, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:00:36.166085 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:36.138978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:49485 (local address 127.11.175.254:41129)
0619 15:00:36.139797 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:36.139816 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:36.140756 (+ 940us) server_negotiation.cc:407] Connection header received
0619 15:00:36.143998 (+ 3242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:36.144017 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:36.144447 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:36.144815 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:36.147554 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:36.149237 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:36.151089 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:36.152289 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:36.152508 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:36.156796 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:36.156805 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:36.156822 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:36.156871 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:36.160309 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:36.161639 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:36.161651 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:36.161663 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:36.161870 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:36.162567 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:36.162572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:36.162574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:36.163272 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:36.163454 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:36.164677 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0619 15:00:36.165070 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":313,"threads_started":1}
I20260619 15:00:36.167003 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:36.139110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:49485)
0619 15:00:36.140264 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:36.140329 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:36.143304 (+ 2975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:36.145076 (+ 1772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:36.145091 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:36.146031 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:00:36.146997 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:36.147032 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:36.149462 (+ 2430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:36.149467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:36.150913 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:36.150929 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:36.152527 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:36.152532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:36.152736 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:36.153482 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:00:36.153514 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:00:36.156376 (+ 2862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:00:36.160516 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:36.160529 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:36.160551 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:36.161322 (+ 771us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:36.162021 (+ 699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:36.162030 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:36.162035 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:36.162379 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:36.163439 (+ 1060us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:00:36.163452 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:00:36.164009 (+ 557us) client_negotiation.cc:769] Sending connection context
0619 15:00:36.164418 (+ 409us) client_negotiation.cc:240] Negotiation successful
0619 15:00:36.165259 (+ 841us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":570,"threads_started":1}
I20260619 15:00:36.170923 28229 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41129
I20260619 15:00:36.171350 28229 heartbeater.cc:464] Registering TS with master...
I20260619 15:00:36.172300 28229 heartbeater.cc:511] Master 127.11.175.254:41129 requested a full tablet report, sending...
I20260619 15:00:36.175127 28043 ts_manager.cc:194] Registered new tserver with Master: e6cc69b514fe4fbba619375613fe8120 (127.11.175.193:43379)
I20260619 15:00:36.177273 28043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:49485
W20260619 15:00:36.510040 28233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:36.510352 28233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:36.510450 28233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:36.518179 28233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:36.518359 28233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:36.518489 28233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 15:00:36.528995 28233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:45713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=45801
--tserver_master_addrs=127.11.175.254:41129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:36.530336 28233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:36.531788 28233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:36.546105 28238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:36.546170 28241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:36.546147 28239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:36.547204 28233 server_base.cc:1034] running on GCE node
I20260619 15:00:36.548014 28233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:36.549198 28233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:36.550428 28233 hybrid_clock.cc:648] HybridClock initialized: now 1781881236550346 us; error 73 us; skew 500 ppm
I20260619 15:00:36.550868 28233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:36 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881236, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:36.554301 28233 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 15:00:36.556172 28233 webserver.cc:460] Webserver started at http://127.11.175.194:45801/ using document root <none> and password file <none>
I20260619 15:00:36.556991 28233 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:36.557169 28233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:36.563364 28233 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260619 15:00:36.567582 28248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:36.569617 28233 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260619 15:00:36.569861 28233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "decc6cea2acd4f1c850d066d19a55154"
format_stamp: "Formatted at 2026-06-19 15:00:13 on dist-test-slave-76rk"
server_key: "b124dec6758fb893dc0fe1c2016eafb1"
server_key_iv: "5b824b84b028e1efd7ff2d1ceaace621"
server_key_version: "encryptionkey@0"
I20260619 15:00:36.571072 28233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:36.591463 28233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:36.596740 28233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:36.597066 28233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:36.598210 28233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:00:36.599849 28233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:00:36.599977 28233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:36.600121 28233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:00:36.600217 28233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:36.637614 28233 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:45713
I20260619 15:00:36.637696 28360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:45713 every 8 connection(s)
I20260619 15:00:36.639835 28233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 15:00:36.642129 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 28233
I20260619 15:00:36.642954 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:36139
--local_ip_for_outbound_sockets=127.11.175.195
--tserver_master_addrs=127.11.175.254:41129
--webserver_port=35889
--webserver_interface=127.11.175.195
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 15:00:36 dist-test-slave-76rk krb5kdc[27457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881236, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:00:36.670185 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:36.643262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:57361 (local address 127.11.175.254:41129)
0619 15:00:36.643480 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:36.643487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:36.647004 (+ 3517us) server_negotiation.cc:407] Connection header received
0619 15:00:36.650390 (+ 3386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:36.650395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:36.650510 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:36.650628 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:36.652883 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:36.653629 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:36.656677 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:36.657176 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:36.657301 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:36.663043 (+ 5742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:36.663050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:36.663053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:36.663108 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:36.666088 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:36.667200 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:36.667205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:36.667207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:36.667417 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:36.667983 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:36.667989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:36.667993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:36.668439 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:36.668615 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:36.669553 (+ 938us) server_negotiation.cc:299] Negotiation successful
0619 15:00:36.669826 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 15:00:36.671411 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:36.643925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:57361)
0619 15:00:36.646769 (+ 2844us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:36.646839 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:36.649928 (+ 3089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:36.651001 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:36.651017 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:36.651947 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:00:36.652631 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:36.652652 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:36.653840 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:36.653848 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:36.656378 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:36.656404 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:36.657483 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:36.657493 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:36.657909 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:36.658963 (+ 1054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:00:36.659008 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:00:36.662611 (+ 3603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:00:36.666371 (+ 3760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:36.666389 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:36.666406 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:36.667018 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:36.667551 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:36.667556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:36.667561 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:36.667849 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:36.668584 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:00:36.668593 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:00:36.669106 (+ 513us) client_negotiation.cc:769] Sending connection context
0619 15:00:36.669461 (+ 355us) client_negotiation.cc:240] Negotiation successful
0619 15:00:36.670084 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2388,"thread_start_us":2134,"threads_started":1}
I20260619 15:00:36.673252 28361 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41129
I20260619 15:00:36.673710 28361 heartbeater.cc:464] Registering TS with master...
I20260619 15:00:36.674611 28361 heartbeater.cc:511] Master 127.11.175.254:41129 requested a full tablet report, sending...
I20260619 15:00:36.676510 28043 ts_manager.cc:194] Registered new tserver with Master: decc6cea2acd4f1c850d066d19a55154 (127.11.175.194:45713)
I20260619 15:00:36.677650 28043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:57361
W20260619 15:00:37.033705 28364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:37.034610 28364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:37.034976 28364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:37.043387 28364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:37.043526 28364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:37.043650 28364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 15:00:37.054767 28364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:36139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=35889
--tserver_master_addrs=127.11.175.254:41129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:37.056308 28364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:37.058055 28364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:37.074764 28369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:37.075316 28370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:37.075843 28372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:37.082294 28364 server_base.cc:1034] running on GCE node
I20260619 15:00:37.083209 28364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:37.084367 28364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:37.085686 28364 hybrid_clock.cc:648] HybridClock initialized: now 1781881237085611 us; error 76 us; skew 500 ppm
I20260619 15:00:37.086117 28364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:37 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881237, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:37.100028 28364 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 15:00:37.101908 28364 webserver.cc:460] Webserver started at http://127.11.175.195:35889/ using document root <none> and password file <none>
I20260619 15:00:37.102802 28364 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:37.102933 28364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:37.109373 28364 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 15:00:37.113333 28379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:37.115265 28364 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:00:37.115451 28364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de"
format_stamp: "Formatted at 2026-06-19 15:00:13 on dist-test-slave-76rk"
server_key: "cf3143ed25236ef2c5882aad3dc39237"
server_key_iv: "79b64421c6e6efeea25f9e54a127b200"
server_key_version: "encryptionkey@0"
I20260619 15:00:37.116652 28364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:37.132982 28364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:37.139290 28364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:37.139647 28364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:37.140955 28364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:00:37.142470 28364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:00:37.142565 28364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:37.142724 28364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:00:37.142789 28364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:37.180629 28364 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:36139
I20260619 15:00:37.180727 28491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:36139 every 8 connection(s)
I20260619 15:00:37.181227 28229 heartbeater.cc:503] Master 127.11.175.254:41129 was elected leader, sending a full tablet report...
I20260619 15:00:37.182411 28364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 15:00:37.188620 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 28364
Jun 19 15:00:37 dist-test-slave-76rk krb5kdc[27457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881237, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:00:37.207178 28496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.189617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37674 (local address 127.11.175.254:41129)
0619 15:00:37.190209 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.190217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.191240 (+ 1023us) server_negotiation.cc:407] Connection header received
0619 15:00:37.191438 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.191450 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.191509 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.191621 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:37.193331 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.194151 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.195655 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.196103 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.196258 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.199707 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:37.199714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:37.199719 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:37.199768 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:37.203802 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.204833 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.204839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.204841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.204997 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.205522 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.205526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.205528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.205821 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:37.206071 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.206503 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.206741 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"spinlock_wait_cycles":1920,"thread_start_us":253,"threads_started":1}
I20260619 15:00:37.207324 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.185332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:51231 (local address 127.11.175.254:41129)
0619 15:00:37.185816 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.185820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.186833 (+ 1013us) server_negotiation.cc:407] Connection header received
0619 15:00:37.188926 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.188932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.189034 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.189164 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:37.191107 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.191919 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.193566 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.194051 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.194196 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.198309 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:37.198314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:37.198318 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:37.198352 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:37.202342 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.203514 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.203524 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.203528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.203735 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.204507 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.204511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.204514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.204960 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:37.205144 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.206281 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.206580 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"spinlock_wait_cycles":896,"thread_start_us":261,"threads_started":1}
I20260619 15:00:37.208491 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.185897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:51231)
0619 15:00:37.186613 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.186705 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:37.188560 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:37.189453 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:37.189463 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:37.190236 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:00:37.190924 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.190944 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.192084 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.192094 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.193410 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.193422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.194299 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.194308 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.194564 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.195357 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:00:37.195380 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:00:37.198020 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:00:37.202570 (+ 4550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:37.202581 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:37.202602 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:37.203325 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:37.203972 (+ 647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:00:37.203978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:00:37.203981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:00:37.204325 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:00:37.205182 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:00:37.205192 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:00:37.205712 (+ 520us) client_negotiation.cc:769] Sending connection context
0619 15:00:37.206019 (+ 307us) client_negotiation.cc:240] Negotiation successful
0619 15:00:37.206705 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":286,"threads_started":1}
I20260619 15:00:37.210191 28492 heartbeater.cc:347] Connected to a master server at 127.11.175.254:41129
I20260619 15:00:37.210628 28492 heartbeater.cc:464] Registering TS with master...
I20260619 15:00:37.211536 28492 heartbeater.cc:511] Master 127.11.175.254:41129 requested a full tablet report, sending...
I20260619 15:00:37.213104 28043 ts_manager.cc:194] Registered new tserver with Master: 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de (127.11.175.195:36139)
I20260619 15:00:37.213999 28043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:51231
I20260619 15:00:37.223295 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 15:00:37.231330 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.224477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37684 (local address 127.11.175.254:41129)
0619 15:00:37.224701 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.224711 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.225147 (+ 436us) server_negotiation.cc:407] Connection header received
0619 15:00:37.225376 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.225380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.225523 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.225669 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:00:37.226103 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.226983 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.228676 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.229233 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.229347 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.230026 (+ 679us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:00:37.230419 (+ 393us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 15:00:37.230959 (+ 540us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37684: token expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260619 15:00:37.231494 28494 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37684: token expired
W20260619 15:00:37.232116 27966 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.175.254:41129)
I20260619 15:00:37.247485 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.232748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.11.175.254:41129)
0619 15:00:37.232967 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.232971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.232988 (+ 17us) server_negotiation.cc:407] Connection header received
0619 15:00:37.233125 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.233129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.233204 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.233297 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:37.234905 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.235787 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.237296 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.237712 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.237821 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.240759 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:37.240764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:37.240768 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:37.240808 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:37.243798 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.244874 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.244879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.244882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.245049 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.245660 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.245666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.245669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.246183 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:37.246436 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.246861 (+ 425us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.247117 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260619 15:00:37.251940 28043 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 15:00:37.259891 28043 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 15:00:37.304533 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.278888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:45713 (local address 127.0.0.1:45812)
0619 15:00:37.291962 (+ 13074us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.291985 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:37.292128 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:37.293867 (+ 1739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:37.293880 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:37.293944 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:00:37.294686 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.294709 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.298948 (+ 4239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.298954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.301245 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.301261 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.303578 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.303585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.303829 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.303938 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 15:00:37.304051 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 15:00:37.304224 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12901,"mutex_wait_us":24,"thread_start_us":18279,"threads_started":1}
I20260619 15:00:37.306069 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.279477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45812 (local address 127.11.175.194:45713)
0619 15:00:37.292873 (+ 13396us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.292886 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.292927 (+ 41us) server_negotiation.cc:407] Connection header received
0619 15:00:37.293076 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.293088 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.293486 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.295016 (+ 1530us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:00:37.295315 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.297391 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.301484 (+ 4093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.303292 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.305150 (+ 1858us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.305406 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.305506 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.305704 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13158,"thread_start_us":12999,"threads_started":1}
I20260619 15:00:37.306969 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.279521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46524 (local address 127.11.175.193:43379)
0619 15:00:37.296631 (+ 17110us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.296645 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.297919 (+ 1274us) server_negotiation.cc:407] Connection header received
0619 15:00:37.298033 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.298039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.298266 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.298441 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:00:37.299289 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.300927 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.304037 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.305564 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.305787 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.305959 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.306229 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.306416 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16857,"thread_start_us":17164,"threads_started":1}
I20260619 15:00:37.306999 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.279421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.0.0.1:46524)
0619 15:00:37.297563 (+ 18142us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.297598 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:37.297771 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:37.298488 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:37.298493 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:37.298519 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:00:37.299063 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.299077 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.301088 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.301097 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.303808 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.303823 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.305824 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.305832 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.306017 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.306042 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 15:00:37.306152 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 15:00:37.306289 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17959,"mutex_wait_us":263,"thread_start_us":11255,"threads_started":1}
I20260619 15:00:37.308527 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.279007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42150 (local address 127.11.175.195:36139)
0619 15:00:37.298016 (+ 19009us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.298022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.298050 (+ 28us) server_negotiation.cc:407] Connection header received
0619 15:00:37.298126 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.298132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.298306 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.298522 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:00:37.299593 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.301186 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.303485 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.305231 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.305441 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.305620 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.307821 (+ 2201us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.308119 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18825,"thread_start_us":18661,"threads_started":1}
I20260619 15:00:37.309125 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.278564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:36139 (local address 127.0.0.1:42150)
0619 15:00:37.291151 (+ 12587us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.291220 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:37.291437 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:37.298549 (+ 7112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:37.298556 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:37.298589 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:00:37.299387 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.299402 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.301360 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.301367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.303300 (+ 1933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.303315 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.305477 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.305484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.305670 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.305718 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 15:00:37.305826 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 15:00:37.305988 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12308,"mutex_wait_us":45,"thread_start_us":13031,"threads_started":1}
I20260619 15:00:37.311313 28296 tablet_service.cc:1467] Processing CreateTablet for tablet 10c3bbf420ec4e809455331384a3b899 (DEFAULT_TABLE table=test-table [id=1ea667a9da9a4582b4f239d4f199716f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:00:37.311458 28164 tablet_service.cc:1467] Processing CreateTablet for tablet 10c3bbf420ec4e809455331384a3b899 (DEFAULT_TABLE table=test-table [id=1ea667a9da9a4582b4f239d4f199716f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:00:37.312949 28296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10c3bbf420ec4e809455331384a3b899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:00:37.312983 28164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10c3bbf420ec4e809455331384a3b899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:00:37.313382 28427 tablet_service.cc:1467] Processing CreateTablet for tablet 10c3bbf420ec4e809455331384a3b899 (DEFAULT_TABLE table=test-table [id=1ea667a9da9a4582b4f239d4f199716f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:00:37.314947 28427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10c3bbf420ec4e809455331384a3b899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:00:37.326515 28509 tablet_bootstrap.cc:492] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Bootstrap starting.
I20260619 15:00:37.329421 28510 tablet_bootstrap.cc:492] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: Bootstrap starting.
I20260619 15:00:37.330871 28509 tablet_bootstrap.cc:654] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Neither blocks nor log segments found. Creating new log.
I20260619 15:00:37.332294 28511 tablet_bootstrap.cc:492] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Bootstrap starting.
I20260619 15:00:37.332484 28509 log.cc:824] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Log is configured to *not* fsync() on all Append() calls
I20260619 15:00:37.333350 28510 tablet_bootstrap.cc:654] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: Neither blocks nor log segments found. Creating new log.
I20260619 15:00:37.334959 28510 log.cc:824] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: Log is configured to *not* fsync() on all Append() calls
I20260619 15:00:37.335445 28511 tablet_bootstrap.cc:654] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Neither blocks nor log segments found. Creating new log.
I20260619 15:00:37.336433 28511 log.cc:824] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Log is configured to *not* fsync() on all Append() calls
I20260619 15:00:37.338037 28509 tablet_bootstrap.cc:492] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: No bootstrap required, opened a new log
I20260619 15:00:37.338357 28509 ts_tablet_manager.cc:1397] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260619 15:00:37.340165 28511 tablet_bootstrap.cc:492] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: No bootstrap required, opened a new log
I20260619 15:00:37.340407 28510 tablet_bootstrap.cc:492] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: No bootstrap required, opened a new log
I20260619 15:00:37.340473 28511 ts_tablet_manager.cc:1397] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260619 15:00:37.340787 28510 ts_tablet_manager.cc:1397] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260619 15:00:37.347949 28511 raft_consensus.cc:348] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } }
I20260619 15:00:37.348388 28511 raft_consensus.cc:374] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:00:37.348490 28511 raft_consensus.cc:729] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de, State: Initialized, Role: FOLLOWER
I20260619 15:00:37.349160 28511 consensus_queue.cc:260] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } }
I20260619 15:00:37.350049 28509 raft_consensus.cc:348] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } }
I20260619 15:00:37.350539 28509 raft_consensus.cc:374] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:00:37.350656 28509 raft_consensus.cc:729] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6cc69b514fe4fbba619375613fe8120, State: Initialized, Role: FOLLOWER
I20260619 15:00:37.351317 28492 heartbeater.cc:503] Master 127.11.175.254:41129 was elected leader, sending a full tablet report...
I20260619 15:00:37.351377 28509 consensus_queue.cc:260] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } }
I20260619 15:00:37.351716 28511 ts_tablet_manager.cc:1428] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Time spent starting tablet: real 0.011s user 0.006s sys 0.005s
I20260619 15:00:37.353740 28510 raft_consensus.cc:348] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } }
I20260619 15:00:37.354333 28510 raft_consensus.cc:374] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:00:37.354465 28510 raft_consensus.cc:729] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: decc6cea2acd4f1c850d066d19a55154, State: Initialized, Role: FOLLOWER
I20260619 15:00:37.355264 28510 consensus_queue.cc:260] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } }
I20260619 15:00:37.356637 28361 heartbeater.cc:503] Master 127.11.175.254:41129 was elected leader, sending a full tablet report...
I20260619 15:00:37.357724 28510 ts_tablet_manager.cc:1428] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260619 15:00:37.359503 28509 ts_tablet_manager.cc:1428] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Time spent starting tablet: real 0.021s user 0.012s sys 0.008s
W20260619 15:00:37.388605 28230 tablet.cc:2369] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 15:00:37.392654 28362 tablet.cc:2369] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:00:37.414232 28517 raft_consensus.cc:482] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 15:00:37.414603 28517 raft_consensus.cc:504] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } }
I20260619 15:00:37.416369 28517 leader_election.cc:288] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6cc69b514fe4fbba619375613fe8120 (127.11.175.193:43379), 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de (127.11.175.195:36139)
I20260619 15:00:37.426386 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.416971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:56769 (local address 127.11.175.193:43379)
0619 15:00:37.417193 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.417200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.418031 (+ 831us) server_negotiation.cc:407] Connection header received
0619 15:00:37.418404 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.418409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.418483 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.418583 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:00:37.419618 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.420863 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.424171 (+ 3308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.425453 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.425631 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.425708 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.425972 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.426107 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 15:00:37.426586 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.417096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:56331 (local address 127.11.175.195:36139)
0619 15:00:37.417302 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.417308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.418030 (+ 722us) server_negotiation.cc:407] Connection header received
0619 15:00:37.418385 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.418392 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.418478 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.418583 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:00:37.419610 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.420858 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.424150 (+ 3292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.425451 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.425633 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.425708 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.426187 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.426324 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 15:00:37.426890 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.416952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.11.175.194:56769)
0619 15:00:37.417862 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.417890 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:37.418092 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:37.418632 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:37.418690 (+ 58us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:37.418722 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:00:37.419442 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.419455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.421003 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.421007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.423923 (+ 2916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.423940 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.425673 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.425678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.425796 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.425827 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 15:00:37.425943 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 15:00:37.426090 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":215,"spinlock_wait_cycles":39552,"thread_start_us":485,"threads_started":1}
I20260619 15:00:37.426921 28519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.416956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:36139 (local address 127.11.175.194:56331)
0619 15:00:37.417883 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.417903 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 15:00:37.418092 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:00:37.418656 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:00:37.418662 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:00:37.418712 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:00:37.419322 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.419339 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.421001 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.421006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.423922 (+ 2916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:00:37.423940 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.425875 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.425880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:00:37.426017 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.426041 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 15:00:37.426159 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 15:00:37.426297 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"mutex_wait_us":296,"spinlock_wait_cycles":31872,"thread_start_us":533,"threads_started":1}
I20260619 15:00:37.427954 28447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c3bbf420ec4e809455331384a3b899" candidate_uuid: "decc6cea2acd4f1c850d066d19a55154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" is_pre_election: true
I20260619 15:00:37.427959 28184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c3bbf420ec4e809455331384a3b899" candidate_uuid: "decc6cea2acd4f1c850d066d19a55154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6cc69b514fe4fbba619375613fe8120" is_pre_election: true
I20260619 15:00:37.428460 28447 raft_consensus.cc:2454] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate decc6cea2acd4f1c850d066d19a55154 in term 0.
I20260619 15:00:37.428460 28184 raft_consensus.cc:2454] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate decc6cea2acd4f1c850d066d19a55154 in term 0.
I20260619 15:00:37.429143 28249 leader_election.cc:302] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de, decc6cea2acd4f1c850d066d19a55154, e6cc69b514fe4fbba619375613fe8120; no voters:
I20260619 15:00:37.429591 28517 raft_consensus.cc:2781] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 15:00:37.429744 28517 raft_consensus.cc:482] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 15:00:37.429865 28517 raft_consensus.cc:3037] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:00:37.432353 28517 raft_consensus.cc:504] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } }
I20260619 15:00:37.433030 28517 leader_election.cc:288] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [CANDIDATE]: Term 1 election: Requested vote from peers e6cc69b514fe4fbba619375613fe8120 (127.11.175.193:43379), 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de (127.11.175.195:36139)
I20260619 15:00:37.433640 28447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c3bbf420ec4e809455331384a3b899" candidate_uuid: "decc6cea2acd4f1c850d066d19a55154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de"
I20260619 15:00:37.433663 28184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c3bbf420ec4e809455331384a3b899" candidate_uuid: "decc6cea2acd4f1c850d066d19a55154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6cc69b514fe4fbba619375613fe8120"
I20260619 15:00:37.433887 28447 raft_consensus.cc:3037] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:00:37.433893 28184 raft_consensus.cc:3037] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120 [term 0 FOLLOWER]: Advancing to term 1
W20260619 15:00:37.434576 28493 tablet.cc:2369] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:00:37.437427 28447 raft_consensus.cc:2454] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate decc6cea2acd4f1c850d066d19a55154 in term 1.
I20260619 15:00:37.437427 28184 raft_consensus.cc:2454] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate decc6cea2acd4f1c850d066d19a55154 in term 1.
I20260619 15:00:37.437994 28251 leader_election.cc:302] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de, decc6cea2acd4f1c850d066d19a55154; no voters:
I20260619 15:00:37.438349 28517 raft_consensus.cc:2781] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:00:37.438905 28517 raft_consensus.cc:686] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 1 LEADER]: Becoming Leader. State: Replica: decc6cea2acd4f1c850d066d19a55154, State: Running, Role: LEADER
I20260619 15:00:37.439419 28517 consensus_queue.cc:237] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } } peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } }
I20260619 15:00:37.444864 28042 catalog_manager.cc:5495] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 reported cstate change: term changed from 0 to 1, leader changed from <none> to decc6cea2acd4f1c850d066d19a55154 (127.11.175.194). New cstate: current_term: 1 leader_uuid: "decc6cea2acd4f1c850d066d19a55154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "decc6cea2acd4f1c850d066d19a55154" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45713 } health_report { overall_health: HEALTHY } } }
I20260619 15:00:37.471750 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.464640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45822 (local address 127.11.175.194:45713)
0619 15:00:37.464889 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.464896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.464994 (+ 98us) server_negotiation.cc:407] Connection header received
0619 15:00:37.465331 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.465336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.465422 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.465596 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:00:37.465986 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.466994 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.469152 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.469575 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.469733 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.470296 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:00:37.470998 (+ 702us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:00:37.471166 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.471335 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.471440 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260619 15:00:37.482228 28184 raft_consensus.cc:1261] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120 [term 1 FOLLOWER]: Refusing update from remote peer decc6cea2acd4f1c850d066d19a55154: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:00:37.482347 28447 raft_consensus.cc:1261] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 1 FOLLOWER]: Refusing update from remote peer decc6cea2acd4f1c850d066d19a55154: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:00:37.483466 28520 consensus_queue.cc:1035] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:00:37.484050 28517 consensus_queue.cc:1035] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6cc69b514fe4fbba619375613fe8120" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 43379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:00:37.499734 28524 mvcc.cc:205] Tried to move back new op lower bound from 7298585548715118592 to 7298585548556341248. Current Snapshot: MvccSnapshot[applied={T|T < 7298585548715118592}]
I20260619 15:00:37.513443 28042 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:00:37.513797 28042 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:00:37.518091 28042 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Sending DeleteTablet for 3 replicas of tablet 10c3bbf420ec4e809455331384a3b899
I20260619 15:00:37.519541 28164 tablet_service.cc:1514] Processing DeleteTablet for tablet 10c3bbf420ec4e809455331384a3b899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:00:37 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:46524
I20260619 15:00:37.519970 28296 tablet_service.cc:1514] Processing DeleteTablet for tablet 10c3bbf420ec4e809455331384a3b899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:00:37 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:45812
I20260619 15:00:37.520152 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 28080
I20260619 15:00:37.520340 28427 tablet_service.cc:1514] Processing DeleteTablet for tablet 10c3bbf420ec4e809455331384a3b899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:00:37 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:42150
I20260619 15:00:37.532791 28534 tablet_replica.cc:330] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: stopping tablet replica
I20260619 15:00:37.533383 28534 raft_consensus.cc:2229] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 1 LEADER]: Raft consensus shutting down.
I20260619 15:00:37.534220 28534 raft_consensus.cc:2258] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:00:37.535310 28535 tablet_replica.cc:330] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: stopping tablet replica
I20260619 15:00:37.535833 28535 raft_consensus.cc:2229] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:00:37.536412 28535 raft_consensus.cc:2258] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:00:37.537345 28533 tablet_replica.cc:330] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: stopping tablet replica
I20260619 15:00:37.537844 28533 raft_consensus.cc:2229] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:00:37.538450 28533 raft_consensus.cc:2258] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:00:37.540827 28534 ts_tablet_manager.cc:1905] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:00:37.541702 28535 ts_tablet_manager.cc:1905] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:00:37.542011 28533 ts_tablet_manager.cc:1905] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:00:37.546217 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.521455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46540 (local address 127.11.175.193:43379)
0619 15:00:37.521643 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.521650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.521968 (+ 318us) server_negotiation.cc:407] Connection header received
0619 15:00:37.522057 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.522066 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.522172 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.522284 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:37.524945 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.526582 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.529538 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.530617 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.530815 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.535420 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:37.535442 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:37.535463 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:37.535521 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:37.540711 (+ 5190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.542104 (+ 1393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.542117 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.542124 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.542339 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.543042 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.543053 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.543056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.543639 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:37.544033 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.545496 (+ 1463us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.545794 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 15:00:37.547282 28224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:37.550824 28534 ts_tablet_manager.cc:1918] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:00:37.551157 28534 log.cc:1196] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/10c3bbf420ec4e809455331384a3b899
I20260619 15:00:37.551962 28534 ts_tablet_manager.cc:1939] T 10c3bbf420ec4e809455331384a3b899 P decc6cea2acd4f1c850d066d19a55154: Deleting consensus metadata
I20260619 15:00:37.554558 28030 catalog_manager.cc:4841] TS decc6cea2acd4f1c850d066d19a55154 (127.11.175.194:45713): tablet 10c3bbf420ec4e809455331384a3b899 (table test-table [id=1ea667a9da9a4582b4f239d4f199716f]) successfully deleted
I20260619 15:00:37.554585 28535 ts_tablet_manager.cc:1918] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:00:37.555913 28535 log.cc:1196] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/10c3bbf420ec4e809455331384a3b899
I20260619 15:00:37.556677 28535 ts_tablet_manager.cc:1939] T 10c3bbf420ec4e809455331384a3b899 P 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de: Deleting consensus metadata
I20260619 15:00:37.559422 28029 catalog_manager.cc:4841] TS 2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de (127.11.175.195:36139): tablet 10c3bbf420ec4e809455331384a3b899 (table test-table [id=1ea667a9da9a4582b4f239d4f199716f]) successfully deleted
I20260619 15:00:37.716746 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 28080
W20260619 15:00:37.743005 28027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.175.193:43379 (error 108)
I20260619 15:00:37.743116 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 28233
I20260619 15:00:37.745728 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.745188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.0.0.1:46546)
0619 15:00:37.745389 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.745525 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260619 15:00:37.746054 28027 catalog_manager.cc:4584] TS e6cc69b514fe4fbba619375613fe8120 (127.11.175.193:43379): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 10c3bbf420ec4e809455331384a3b899: Network error: Client connection negotiation failed: client connection to 127.11.175.193:43379: connect: Connection refused (error 111)
I20260619 15:00:37.762835 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.744852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45826 (local address 127.11.175.194:45713)
0619 15:00:37.745054 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.745058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.745078 (+ 20us) server_negotiation.cc:407] Connection header received
0619 15:00:37.745135 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.745139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.745218 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.745356 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:37.747442 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.748758 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.750194 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.750933 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.751079 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.754114 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:37.754141 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:37.754163 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:37.754233 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:37.758857 (+ 4624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.759961 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.759973 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.759976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.760187 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.760790 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.760796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.760800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.761365 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:37.761763 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.762083 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.762361 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 15:00:37.763712 28356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:37.799925 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.799358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.0.0.1:46552)
0619 15:00:37.799618 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.799762 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260619 15:00:37.801151 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.800738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.0.0.1:46560)
0619 15:00:37.800903 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.801001 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 15:00:37.862354 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.861614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.0.0.1:46574)
0619 15:00:37.862079 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.862196 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279}
I20260619 15:00:37.863577 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.863207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.0.0.1:46590)
0619 15:00:37.863366 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.863449 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 15:00:37.885052 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 28233
I20260619 15:00:37.905159 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 28364
I20260619 15:00:37.921285 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.905938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42166 (local address 127.11.175.195:36139)
0619 15:00:37.906108 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 15:00:37.906112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 15:00:37.906132 (+ 20us) server_negotiation.cc:407] Connection header received
0619 15:00:37.906352 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:00:37.906356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:00:37.906425 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:00:37.906519 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:00:37.908114 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.908994 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.910583 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:00:37.911340 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:00:37.911459 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:00:37.914540 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:00:37.914556 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:00:37.914571 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:00:37.914628 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:00:37.917613 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.918502 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.918511 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.918513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.918741 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:00:37.919343 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:00:37.919347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:00:37.919350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:00:37.919781 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:00:37.920028 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0619 15:00:37.920576 (+ 548us) server_negotiation.cc:299] Negotiation successful
0619 15:00:37.920796 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 15:00:37.922091 28487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:37.944376 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.943741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.0.0.1:46592)
0619 15:00:37.944018 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.944140 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260619 15:00:37.945757 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:37.945271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.0.0.1:46596)
0619 15:00:37.945501 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:37.945598 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260619 15:00:38.018870 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 28364
I20260619 15:00:38.036792 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 28010
I20260619 15:00:38.038269 28073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:00:38.159943 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:38.159232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:43379 (local address 127.0.0.1:46612)
0619 15:00:38.159592 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:38.159702 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260619 15:00:38.160029 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 28010
I20260619 15:00:58.184335 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41129
--webserver_interface=127.11.175.254
--webserver_port=41931
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:41129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:00:58.545099 28541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:58.545440 28541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:58.545624 28541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:58.553665 28541 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 15:00:58.553802 28541 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 15:00:58.553848 28541 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 15:00:58.553941 28541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:00:58.554055 28541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:58.554185 28541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:58.554279 28541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:00:58.554368 28541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:00:58.565192 28541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:41129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=41931
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:58.566735 28541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:58.568279 28541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:58.581974 28546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:58.582050 28547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:58.582020 28549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:58.582522 28541 server_base.cc:1034] running on GCE node
I20260619 15:00:58.583580 28541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:58.585245 28541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:58.586560 28541 hybrid_clock.cc:648] HybridClock initialized: now 1781881258586465 us; error 63 us; skew 500 ppm
I20260619 15:00:58.587005 28541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:58 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881258, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:58.596598 28541 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:00:58.598527 28541 webserver.cc:460] Webserver started at http://127.11.175.254:41931/ using document root <none> and password file <none>
I20260619 15:00:58.599341 28541 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:58.599488 28541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:58.605563 28541 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 15:00:58.608917 28556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:58.610613 28541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 15:00:58.610921 28541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "a29bdcdf19004ad5a67ca8410c28bb22"
format_stamp: "Formatted at 2026-06-19 15:00:12 on dist-test-slave-76rk"
server_key: "10a88df01af4fd285c1699ad7a0dd49b"
server_key_iv: "2eda5c3b1c584c844959fc826fd183bb"
server_key_version: "encryptionkey@0"
I20260619 15:00:58.611968 28541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:58.640326 28541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:58.645480 28541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:58.645823 28541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:58.666289 28541 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:41129
I20260619 15:00:58.666337 28607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:41129 every 8 connection(s)
I20260619 15:00:58.668076 28541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:00:58.672353 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 28541
I20260619 15:00:58.673475 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:43379
--local_ip_for_outbound_sockets=127.11.175.193
--tserver_master_addrs=127.11.175.254:41129
--webserver_port=38239
--webserver_interface=127.11.175.193
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 15:00:58.677610 28608 sys_catalog.cc:263] Verifying existing consensus state
I20260619 15:00:58.684094 28608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Bootstrap starting.
I20260619 15:00:58.708367 28608 log.cc:824] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Log is configured to *not* fsync() on all Append() calls
I20260619 15:00:58.726936 28608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 15:00:58.727545 28608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Bootstrap complete.
I20260619 15:00:58.737464 28608 raft_consensus.cc:348] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:58.738446 28608 raft_consensus.cc:729] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a29bdcdf19004ad5a67ca8410c28bb22, State: Initialized, Role: FOLLOWER
I20260619 15:00:58.739181 28608 consensus_queue.cc:260] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:58.739446 28608 raft_consensus.cc:388] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:00:58.739584 28608 raft_consensus.cc:482] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:00:58.739799 28608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 2 FOLLOWER]: Advancing to term 3
I20260619 15:00:58.744438 28608 raft_consensus.cc:504] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:58.745031 28608 leader_election.cc:302] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a29bdcdf19004ad5a67ca8410c28bb22; no voters:
I20260619 15:00:58.745783 28608 leader_election.cc:288] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 15:00:58.745966 28612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 15:00:58.747227 28612 raft_consensus.cc:686] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [term 3 LEADER]: Becoming Leader. State: Replica: a29bdcdf19004ad5a67ca8410c28bb22, State: Running, Role: LEADER
I20260619 15:00:58.747922 28612 consensus_queue.cc:237] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } }
I20260619 15:00:58.748351 28608 sys_catalog.cc:564] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:00:58.750808 28613 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } } }
I20260619 15:00:58.751232 28613 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: This master's current role is: LEADER
I20260619 15:00:58.750589 28614 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a29bdcdf19004ad5a67ca8410c28bb22. Latest consensus state: current_term: 3 leader_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29bdcdf19004ad5a67ca8410c28bb22" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41129 } } }
I20260619 15:00:58.752013 28614 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22 [sys.catalog]: This master's current role is: LEADER
I20260619 15:00:58.760063 28617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:00:58.772540 28617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:00:58.775151 28617 catalog_manager.cc:1259] Loaded cluster ID: 3cb95f960a2f416c933708481afb4950
I20260619 15:00:58.775300 28617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:00:58.783577 28617 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:00:58.786070 28617 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Loaded TSK: 0
I20260619 15:00:58.786419 28617 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a29bdcdf19004ad5a67ca8410c28bb22: Loaded TSK: 1
F20260619 15:00:58.787899 28617 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f71a81229ad google::LogMessage::Fail() at ??:0
I20260619 15:00:58.792521 28629 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f71a812588a google::LogMessage::SendToLog() at ??:0
@ 0x7f71a81224de google::LogMessage::Flush() at ??:0
@ 0x7f71a8123a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f71abe867d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f71abe7573e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f71bdb511c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f71bdb50783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f71bdbdc2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f71bdbdc0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f71bdc265e9 std::function<>::operator()() at ??:0
@ 0x7f71bdb54cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f71bdb52c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f71bdbda610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f71bdbda439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f71aad6e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f71aad7191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f71aad71799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f71aad4ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f71ae0266db start_thread at ??:0
@ 0x7f71a6fbc61f clone at ??:0
W20260619 15:00:59.126091 28610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:59.126405 28610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:59.126504 28610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:59.134289 28610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:59.134418 28610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:59.134534 28610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 15:00:59.144665 28610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:43379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=38239
--tserver_master_addrs=127.11.175.254:41129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:59.146162 28610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:59.148175 28610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:59.164143 28655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:59.164422 28656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:59.164888 28658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:59.165215 28610 server_base.cc:1034] running on GCE node
I20260619 15:00:59.165939 28610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:59.167163 28610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:59.168447 28610 hybrid_clock.cc:648] HybridClock initialized: now 1781881259168394 us; error 51 us; skew 500 ppm
I20260619 15:00:59.168860 28610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:59 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881259, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:59.172664 28610 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 15:00:59.174454 28610 webserver.cc:460] Webserver started at http://127.11.175.193:38239/ using document root <none> and password file <none>
I20260619 15:00:59.175251 28610 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:59.175410 28610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:59.181473 28610 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 15:00:59.185415 28665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:59.187471 28610 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 15:00:59.187652 28610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "e6cc69b514fe4fbba619375613fe8120"
format_stamp: "Formatted at 2026-06-19 15:00:12 on dist-test-slave-76rk"
server_key: "8fc0a7f2e6fe46c1cc345e6a5e20517f"
server_key_iv: "716c91d406a28762f74690744688cee4"
server_key_version: "encryptionkey@0"
I20260619 15:00:59.188714 28610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:59.208376 28610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:59.213368 28610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:59.213657 28610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:59.214844 28610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:00:59.217131 28672 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260619 15:00:59.223390 28672 ts_tablet_manager.cc:1852] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260619 15:00:59.224514 28672 ts_tablet_manager.cc:1905] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:00:59.232888 28672 ts_tablet_manager.cc:1918] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:00:59.233321 28672 log.cc:1196] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/10c3bbf420ec4e809455331384a3b899
I20260619 15:00:59.233970 28672 ts_tablet_manager.cc:1939] T 10c3bbf420ec4e809455331384a3b899 P e6cc69b514fe4fbba619375613fe8120: Deleting consensus metadata
I20260619 15:00:59.234436 28610 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260619 15:00:59.234712 28610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.001s sys 0.000s
I20260619 15:00:59.234912 28610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:00:59.235023 28610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:59.273360 28610 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:43379
I20260619 15:00:59.273459 28778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:43379 every 8 connection(s)
I20260619 15:00:59.274811 28610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 15:00:59.278247 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 28610
I20260619 15:00:59.279332 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:45713
--local_ip_for_outbound_sockets=127.11.175.194
--tserver_master_addrs=127.11.175.254:41129
--webserver_port=45801
--webserver_interface=127.11.175.194
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 15:00:59.280872 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:59.278450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:36601)
0619 15:00:59.279528 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:59.279897 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":397,"threads_started":1}
W20260619 15:00:59.281580 28779 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:41129 (0 consecutive failures): Network error: Failed to ping master at 127.11.175.254:41129: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
I20260619 15:00:59.282575 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:59.282179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:56173)
0619 15:00:59.282363 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:59.282450 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 15:00:59.283663 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:59.283282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:47031)
0619 15:00:59.283441 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:59.283531 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 15:00:59.283907 28779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 15:00:59.633024 28782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:00:59.633363 28782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:00:59.633468 28782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:00:59.641019 28782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:00:59.641155 28782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:00:59.641273 28782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 15:00:59.652093 28782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:45713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=45801
--tserver_master_addrs=127.11.175.254:41129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:00:59.653440 28782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:00:59.654935 28782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:00:59.669579 28788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:59.669520 28790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:00:59.669633 28787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:00:59.671084 28782 server_base.cc:1034] running on GCE node
I20260619 15:00:59.671882 28782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:00:59.673000 28782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:00:59.674259 28782 hybrid_clock.cc:648] HybridClock initialized: now 1781881259674207 us; error 44 us; skew 500 ppm
I20260619 15:00:59.674706 28782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:00:59 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881259, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:00:59.678377 28782 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 15:00:59.680212 28782 webserver.cc:460] Webserver started at http://127.11.175.194:45801/ using document root <none> and password file <none>
I20260619 15:00:59.680968 28782 fs_manager.cc:359] Metadata directory not provided
I20260619 15:00:59.681124 28782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:00:59.687489 28782 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 15:00:59.690698 28797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:59.692479 28782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 15:00:59.692695 28782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "decc6cea2acd4f1c850d066d19a55154"
format_stamp: "Formatted at 2026-06-19 15:00:13 on dist-test-slave-76rk"
server_key: "b124dec6758fb893dc0fe1c2016eafb1"
server_key_iv: "5b824b84b028e1efd7ff2d1ceaace621"
server_key_version: "encryptionkey@0"
I20260619 15:00:59.693960 28782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:00:59.707995 28782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:00:59.713203 28782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:00:59.713537 28782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:00:59.715045 28782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:00:59.716555 28782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:00:59.716856 28782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:59.717017 28782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:00:59.717121 28782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:00:59.752969 28782 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:45713
I20260619 15:00:59.753075 28909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:45713 every 8 connection(s)
I20260619 15:00:59.754528 28782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 15:00:59.757552 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 28782
I20260619 15:00:59.758453 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:36139
--local_ip_for_outbound_sockets=127.11.175.195
--tserver_master_addrs=127.11.175.254:41129
--webserver_port=35889
--webserver_interface=127.11.175.195
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 15:00:59.760504 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:59.758310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:58573)
0619 15:00:59.759315 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:59.759572 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":456,"threads_started":1}
W20260619 15:00:59.761356 28910 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:41129 (0 consecutive failures): Network error: Failed to ping master at 127.11.175.254:41129: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
I20260619 15:00:59.762619 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:59.762146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:39215)
0619 15:00:59.762355 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:59.762462 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260619 15:00:59.763826 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:00:59.763390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:33491)
0619 15:00:59.763568 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 15:00:59.763691 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260619 15:00:59.764091 28910 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 15:01:00.128337 28913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:00.128664 28913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:00.128770 28913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:00.136409 28913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:00.136543 28913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:00.136662 28913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 15:01:00.147161 28913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:36139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=35889
--tserver_master_addrs=127.11.175.254:41129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:00.148523 28913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:00.149897 28913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:00.164844 28918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:00.164742 28919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:00.164839 28921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:00.165526 28913 server_base.cc:1034] running on GCE node
I20260619 15:01:00.166426 28913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:00.167614 28913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:00.168851 28913 hybrid_clock.cc:648] HybridClock initialized: now 1781881260168777 us; error 63 us; skew 500 ppm
I20260619 15:01:00.169214 28913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:00 dist-test-slave-76rk krb5kdc[27457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881260, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:00.172863 28913 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 15:01:00.174719 28913 webserver.cc:460] Webserver started at http://127.11.175.195:35889/ using document root <none> and password file <none>
I20260619 15:01:00.175501 28913 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:00.175671 28913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:00.181634 28913 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260619 15:01:00.184942 28928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:00.186993 28913 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:01:00.187260 28913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "2d1a67e2e2dc4fc7b5ac3b2fd3d6e1de"
format_stamp: "Formatted at 2026-06-19 15:00:13 on dist-test-slave-76rk"
server_key: "cf3143ed25236ef2c5882aad3dc39237"
server_key_iv: "79b64421c6e6efeea25f9e54a127b200"
server_key_version: "encryptionkey@0"
I20260619 15:01:00.188385 28913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:00.212568 28913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:00.218333 28913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:00.218705 28913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:00.219837 28913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:00.221469 28913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:00.221612 28913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:00.221746 28913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:00.221840 28913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:00.258908 28913 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:36139
I20260619 15:01:00.259002 29040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:36139 every 8 connection(s)
I20260619 15:01:00.260497 28913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 15:01:00.265991 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:00.263918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:39975)
0619 15:01:00.264811 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:00.265069 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":371,"threads_started":1}
W20260619 15:01:00.266830 29041 heartbeater.cc:644] Failed to heartbeat to 127.11.175.254:41129 (0 consecutive failures): Network error: Failed to ping master at 127.11.175.254:41129: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
I20260619 15:01:00.267964 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:00.267513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:37637)
0619 15:01:00.267696 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:00.267810 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 15:01:00.267938 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 28913
I20260619 15:01:00.268966 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:00.268552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:49221)
0619 15:01:00.268752 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:00.268846 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260619 15:01:00.269372 29041 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 15:01:00.285496 29045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:00.284631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:49191)
0619 15:01:00.285136 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:00.285247 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":241,"threads_started":1}
I20260619 15:01:00.765725 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:00.764784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:41765)
0619 15:01:00.765341 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:00.765473 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":253,"threads_started":1}
I20260619 15:01:01.271044 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:01.270026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:50291)
0619 15:01:01.270574 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:01.270778 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":270,"threads_started":1}
I20260619 15:01:01.287546 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:01.286766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:46087)
0619 15:01:01.287283 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:01.287371 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":290,"threads_started":1}
I20260619 15:01:01.767980 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:01.767039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:41691)
0619 15:01:01.767574 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:01.767716 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":262,"threads_started":1}
I20260619 15:01:02.273101 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:02.272286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:43657)
0619 15:01:02.272833 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:02.272925 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":257,"threads_started":1}
I20260619 15:01:02.289788 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:02.288895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:49197)
0619 15:01:02.289493 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:02.289596 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":316,"threads_started":1}
I20260619 15:01:02.769831 29052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:02.769032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:60953)
0619 15:01:02.769546 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:02.769651 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":286,"threads_started":1}
I20260619 15:01:03.274952 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:03.274087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:33673)
0619 15:01:03.274578 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:03.274721 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":245,"threads_started":1}
I20260619 15:01:03.291895 29054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:03.291027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:34803)
0619 15:01:03.291611 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:03.291710 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":336,"threads_started":1}
I20260619 15:01:03.771967 29055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:03.771066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:55613)
0619 15:01:03.771661 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:03.771758 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":288,"threads_started":1}
I20260619 15:01:04.276932 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:04.276090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:35299)
0619 15:01:04.276595 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:04.276747 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":234,"threads_started":1}
I20260619 15:01:04.294224 29057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:04.293137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:47721)
0619 15:01:04.293848 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:04.293946 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":376,"threads_started":1}
I20260619 15:01:04.774180 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:04.773237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:40509)
0619 15:01:04.773876 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:04.773979 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":317,"threads_started":1}
I20260619 15:01:05.279093 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:05.278102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:43363)
0619 15:01:05.278827 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:05.278917 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":413,"threads_started":1}
I20260619 15:01:05.296166 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:05.295337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:57297)
0619 15:01:05.295873 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:05.295971 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":282,"threads_started":1}
I20260619 15:01:05.776103 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:05.775376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:32893)
0619 15:01:05.775844 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:05.775937 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":246,"threads_started":1}
I20260619 15:01:06.281028 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:06.280252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:41819)
0619 15:01:06.280750 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:06.280854 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":250,"threads_started":1}
I20260619 15:01:06.298257 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:06.297259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:36701)
0619 15:01:06.297932 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:06.298034 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":338,"threads_started":1}
I20260619 15:01:06.778198 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:06.777204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:60895)
0619 15:01:06.777818 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:06.777916 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":244,"threads_started":1}
I20260619 15:01:07.283262 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:07.282260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:47925)
0619 15:01:07.282911 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:07.283009 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":266,"threads_started":1}
I20260619 15:01:07.300375 29066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:07.299439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:58793)
0619 15:01:07.300010 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:07.300178 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":309,"threads_started":1}
I20260619 15:01:07.780225 29067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:07.779414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:40165)
0619 15:01:07.779954 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:07.780053 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":263,"threads_started":1}
I20260619 15:01:08.285089 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:08.284308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:41961)
0619 15:01:08.284840 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:08.284929 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":290,"threads_started":1}
I20260619 15:01:08.302772 29069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:08.301693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:35121)
0619 15:01:08.302349 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:08.302467 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":284,"threads_started":1}
I20260619 15:01:08.782393 29070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:08.781514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:59701)
0619 15:01:08.782062 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:08.782159 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":268,"threads_started":1}
I20260619 15:01:09.287029 29071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:09.286076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:34781)
0619 15:01:09.286733 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:09.286831 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":308,"threads_started":1}
I20260619 15:01:09.304638 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:09.303875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:47083)
0619 15:01:09.304373 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:09.304467 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":243,"threads_started":1}
I20260619 15:01:09.784309 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:09.783500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:60051)
0619 15:01:09.784041 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:09.784138 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":312,"threads_started":1}
I20260619 15:01:10.289069 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:10.288261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:52179)
0619 15:01:10.288817 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:10.288906 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":281,"threads_started":1}
I20260619 15:01:10.306806 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:10.305853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:49543)
0619 15:01:10.306417 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:10.306512 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":341,"threads_started":1}
I20260619 15:01:10.786049 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:10.785325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:43575)
0619 15:01:10.785767 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:10.785861 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":218,"threads_started":1}
I20260619 15:01:11.291497 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:11.290273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:47659)
0619 15:01:11.291094 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:11.291207 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":594,"threads_started":1}
I20260619 15:01:11.308561 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:11.307826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:33493)
0619 15:01:11.308291 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:11.308389 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":239,"threads_started":1}
I20260619 15:01:11.788003 29079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:11.787212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:42087)
0619 15:01:11.787749 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:11.787840 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":267,"threads_started":1}
I20260619 15:01:12.293679 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:12.292641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:42227)
0619 15:01:12.293271 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:12.293403 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":266,"threads_started":1}
I20260619 15:01:12.310796 29081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:12.309788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:56347)
0619 15:01:12.310367 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:12.310487 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":321,"threads_started":1}
I20260619 15:01:12.790017 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:12.789067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:49517)
0619 15:01:12.789703 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:12.789818 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":373,"threads_started":1}
I20260619 15:01:13.295591 29083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:13.294758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:46441)
0619 15:01:13.295248 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:13.295357 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":234,"threads_started":1}
I20260619 15:01:13.312670 29084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:13.311875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:32813)
0619 15:01:13.312379 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:13.312481 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":249,"threads_started":1}
I20260619 15:01:13.792237 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:13.791271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:33175)
0619 15:01:13.791825 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:13.791958 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":247,"threads_started":1}
I20260619 15:01:14.297564 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:14.296738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:48065)
0619 15:01:14.297243 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:14.297337 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":258,"threads_started":1}
I20260619 15:01:14.315200 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:14.314080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:54221)
0619 15:01:14.314877 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:14.314995 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":350,"threads_started":1}
I20260619 15:01:14.794867 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:14.793784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.194:52859)
0619 15:01:14.794422 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:14.794562 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":335,"threads_started":1}
I20260619 15:01:15.277113 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 28610
I20260619 15:01:15.300179 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:15.279329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.11.175.193:43379)
0619 15:01:15.279893 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:15.279924 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:15.279969 (+ 45us) server_negotiation.cc:407] Connection header received
0619 15:01:15.281846 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:15.281858 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:15.282343 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:15.282757 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:15.284391 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:15.285770 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:15.287422 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:15.288244 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:15.288425 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:15.291853 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:15.291863 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:15.291881 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:15.291930 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:15.295966 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:15.296898 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:15.296906 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:15.296913 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:15.297111 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:15.297556 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:15.297560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:15.297562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:15.298343 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:15.298719 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:15.299348 (+ 629us) server_negotiation.cc:299] Negotiation successful
0619 15:01:15.299741 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":737,"threads_started":1}
I20260619 15:01:15.300664 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:15.299053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.195:33725)
0619 15:01:15.300288 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:15.300449 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1103,"thread_start_us":983,"threads_started":1}
I20260619 15:01:15.302115 28774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:15.391522 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:15.390271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41129 (local address 127.11.175.193:39677)
0619 15:01:15.390903 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:15.391138 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.254:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":257,"threads_started":1}
I20260619 15:01:15.391592 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 28610
I20260619 15:01:15.412331 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 28782
I20260619 15:01:15.433701 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:15.413596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.11.175.194:45713)
0619 15:01:15.414288 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:15.414302 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:15.414351 (+ 49us) server_negotiation.cc:407] Connection header received
0619 15:01:15.416143 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:15.416154 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:15.416614 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:15.416974 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:15.418533 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:15.420029 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:15.421637 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:15.422380 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:15.422537 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:15.426036 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:15.426050 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:15.426076 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:15.426140 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:15.429683 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:15.430477 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:15.430487 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:15.430498 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:15.430791 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:15.431227 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:15.431232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:15.431236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:15.431921 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:15.432262 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:15.432858 (+ 596us) server_negotiation.cc:299] Negotiation successful
0619 15:01:15.433390 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":372,"threads_started":1}
I20260619 15:01:15.435348 28905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:15.523751 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 28782
I20260619 15:01:15.541517 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 28913
I20260619 15:01:15.561957 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:15.542706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.11.175.195:36139)
0619 15:01:15.543358 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:15.543373 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:15.543421 (+ 48us) server_negotiation.cc:407] Connection header received
0619 15:01:15.545259 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:15.545272 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:15.545763 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:15.546087 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:15.547984 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:15.549323 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:15.550905 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:15.551664 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:15.551824 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:15.554784 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:15.554799 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:15.554825 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:15.554888 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:15.558079 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:15.558982 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:15.558990 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:15.558999 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:15.559292 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:15.559730 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:15.559734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:15.559736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:15.560397 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:15.560687 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:15.561134 (+ 447us) server_negotiation.cc:299] Negotiation successful
0619 15:01:15.561628 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":313,"threads_started":1}
I20260619 15:01:15.563622 29036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:15.652310 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 28913
2026-06-19T15:01:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65239 ms)
[----------] 30 tests from SecurityITest (302618 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:15 dist-test-slave-76rk krb5kdc[29099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:15 dist-test-slave-76rk krb5kdc[29099](info): set up 2 sockets
Jun 19 15:01:15 dist-test-slave-76rk krb5kdc[29099](info): commencing operation
krb5kdc: starting...
W20260619 15:01:16.778959 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.006s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:16 dist-test-slave-76rk krb5kdc[29099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:01:17.062879 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:45345
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:45345
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:17.417874 29112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:17.418216 29112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:17.418390 29112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:17.426008 29112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:17.426136 29112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:17.426229 29112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:17.426343 29112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:17.426416 29112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:17.438139 29112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:45345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:45345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:17.439627 29112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:17.440984 29112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:17.453446 29118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:17.453446 29117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:17.453752 29120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:17.454861 29112 server_base.cc:1034] running on GCE node
I20260619 15:01:17.455740 29112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:17.457341 29112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:17.458554 29112 hybrid_clock.cc:648] HybridClock initialized: now 1781881277458504 us; error 45 us; skew 500 ppm
I20260619 15:01:17.458986 29112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:17 dist-test-slave-76rk krb5kdc[29099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881277, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:17.468215 29112 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:01:17.470036 29112 webserver.cc:460] Webserver started at http://127.11.175.254:37147/ using document root <none> and password file <none>
I20260619 15:01:17.470911 29112 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:17.471060 29112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:17.471416 29112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:17.474825 29112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "d6cb38a94ef244db9af3c13a7d39c72f"
format_stamp: "Formatted at 2026-06-19 15:01:17 on dist-test-slave-76rk"
server_key: "caaa23744f4f3af4548f44abbe62205e"
server_key_iv: "a744eb26a3d22ad763648a3d029f3008"
server_key_version: "encryptionkey@0"
I20260619 15:01:17.475731 29112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "d6cb38a94ef244db9af3c13a7d39c72f"
format_stamp: "Formatted at 2026-06-19 15:01:17 on dist-test-slave-76rk"
server_key: "caaa23744f4f3af4548f44abbe62205e"
server_key_iv: "a744eb26a3d22ad763648a3d029f3008"
server_key_version: "encryptionkey@0"
I20260619 15:01:17.481958 29112 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 15:01:17.486145 29127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:17.487954 29112 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:01:17.488199 29112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "d6cb38a94ef244db9af3c13a7d39c72f"
format_stamp: "Formatted at 2026-06-19 15:01:17 on dist-test-slave-76rk"
server_key: "caaa23744f4f3af4548f44abbe62205e"
server_key_iv: "a744eb26a3d22ad763648a3d029f3008"
server_key_version: "encryptionkey@0"
I20260619 15:01:17.488443 29112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:17.528987 29112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:17.533735 29112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:17.534071 29112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:17.554571 29112 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:45345
I20260619 15:01:17.554595 29178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:45345 every 8 connection(s)
I20260619 15:01:17.556097 29112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:17.560278 29179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:17.561061 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 29112
I20260619 15:01:17.561362 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:17.562059 11967 external_mini_cluster.cc:1428] Setting key e080095e656510de7ea56e8194480a74
I20260619 15:01:17.575975 29179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f: Bootstrap starting.
Jun 19 15:01:17 dist-test-slave-76rk krb5kdc[29099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:17.579939 29179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:17.581149 29179 log.cc:824] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:17.584348 29179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f: No bootstrap required, opened a new log
I20260619 15:01:17.587396 29182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:17.564231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56594 (local address 127.11.175.254:45345)
0619 15:01:17.565072 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:17.565088 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:17.565132 (+ 44us) server_negotiation.cc:407] Connection header received
0619 15:01:17.566896 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:17.566921 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:17.567348 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:17.567801 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:17.569751 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:17.571210 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:17.573188 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:17.573952 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:17.574099 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:17.579454 (+ 5355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:17.579463 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:17.579479 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:17.579520 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:17.582944 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:17.583741 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:17.583748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:17.583755 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:17.583928 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:17.584359 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:17.584363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:17.584365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:17.584987 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:17.585306 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:17.585763 (+ 457us) server_negotiation.cc:299] Negotiation successful
0619 15:01:17.586252 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":381,"threads_started":1}
I20260619 15:01:17.592893 29179 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6cb38a94ef244db9af3c13a7d39c72f" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45345 } }
I20260619 15:01:17.593444 29179 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:17.593561 29179 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6cb38a94ef244db9af3c13a7d39c72f, State: Initialized, Role: FOLLOWER
I20260619 15:01:17.594264 29179 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6cb38a94ef244db9af3c13a7d39c72f" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45345 } }
I20260619 15:01:17.594581 29179 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:17.594749 29179 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:17.594942 29179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:17.597411 29179 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6cb38a94ef244db9af3c13a7d39c72f" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45345 } }
I20260619 15:01:17.598085 29179 leader_election.cc:302] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6cb38a94ef244db9af3c13a7d39c72f; no voters:
I20260619 15:01:17.598886 29179 leader_election.cc:288] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:17.599012 29184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:17.599664 29184 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [term 1 LEADER]: Becoming Leader. State: Replica: d6cb38a94ef244db9af3c13a7d39c72f, State: Running, Role: LEADER
I20260619 15:01:17.600348 29184 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6cb38a94ef244db9af3c13a7d39c72f" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45345 } }
I20260619 15:01:17.601502 29179 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:17.603884 29186 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6cb38a94ef244db9af3c13a7d39c72f. Latest consensus state: current_term: 1 leader_uuid: "d6cb38a94ef244db9af3c13a7d39c72f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6cb38a94ef244db9af3c13a7d39c72f" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45345 } } }
I20260619 15:01:17.605003 29186 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:17.603856 29185 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6cb38a94ef244db9af3c13a7d39c72f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6cb38a94ef244db9af3c13a7d39c72f" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45345 } } }
I20260619 15:01:17.606787 29185 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f [sys.catalog]: This master's current role is: LEADER
W20260619 15:01:17.615039 29200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 15:01:17.615209 29200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 15:01:17.615389 29193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:17.617266 29193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:17.626823 29193 catalog_manager.cc:1347] Generated new cluster ID: ab85447c51de4eaa8aa1e5160fa199dc
I20260619 15:01:17.627007 29193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:17.651996 29193 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:17.653620 29193 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:17.666013 29193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f: Generated new TSK 0
I20260619 15:01:17.667210 29193 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 15:01:17.742527 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45345
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:18.100550 29205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:18.100914 29205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:18.101090 29205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:18.110004 29205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:18.110219 29205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:18.110415 29205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 15:01:18.121891 29205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:18.123431 29205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:18.124822 29205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:18.139899 29210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:18.139935 29211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:18.139861 29213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:18.141023 29205 server_base.cc:1034] running on GCE node
I20260619 15:01:18.141847 29205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:18.143043 29205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:18.144308 29205 hybrid_clock.cc:648] HybridClock initialized: now 1781881278144232 us; error 75 us; skew 500 ppm
I20260619 15:01:18.144682 29205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:18 dist-test-slave-76rk krb5kdc[29099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881278, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:18.148799 29205 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 15:01:18.150810 29205 webserver.cc:460] Webserver started at http://127.11.175.193:33311/ using document root <none> and password file <none>
I20260619 15:01:18.151638 29205 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:18.151820 29205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:18.152194 29205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:18.155556 29205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "622e52f43d9a413b96e717d3888e0d7b"
format_stamp: "Formatted at 2026-06-19 15:01:18 on dist-test-slave-76rk"
server_key: "6a549f26ce13cf45f3eed56170d74904"
server_key_iv: "3a899185b862ad5d6ceb82793008f61f"
server_key_version: "encryptionkey@0"
I20260619 15:01:18.156399 29205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "622e52f43d9a413b96e717d3888e0d7b"
format_stamp: "Formatted at 2026-06-19 15:01:18 on dist-test-slave-76rk"
server_key: "6a549f26ce13cf45f3eed56170d74904"
server_key_iv: "3a899185b862ad5d6ceb82793008f61f"
server_key_version: "encryptionkey@0"
I20260619 15:01:18.162971 29205 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260619 15:01:18.167101 29220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:18.168705 29205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 15:01:18.168926 29205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "622e52f43d9a413b96e717d3888e0d7b"
format_stamp: "Formatted at 2026-06-19 15:01:18 on dist-test-slave-76rk"
server_key: "6a549f26ce13cf45f3eed56170d74904"
server_key_iv: "3a899185b862ad5d6ceb82793008f61f"
server_key_version: "encryptionkey@0"
I20260619 15:01:18.169167 29205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:18.195228 29205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:18.200582 29205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:18.200965 29205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:18.202206 29205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:18.203897 29205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:18.204010 29205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:18.204177 29205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:18.204253 29205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:18.241050 29205 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:36587
I20260619 15:01:18.241099 29332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:36587 every 8 connection(s)
I20260619 15:01:18.242470 29205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 15:01:18.244117 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 29205
I20260619 15:01:18.244366 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 15:01:18.245040 11967 external_mini_cluster.cc:1428] Setting key 407eb50ce439e56fd9c4ff4b5afd632e
Jun 19 15:01:18 dist-test-slave-76rk krb5kdc[29099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881278, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:18.266986 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:18.245607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:34115 (local address 127.11.175.254:45345)
0619 15:01:18.247170 (+ 1563us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:18.247177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:18.247821 (+ 644us) server_negotiation.cc:407] Connection header received
0619 15:01:18.250171 (+ 2350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:18.250181 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:18.250274 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:18.250402 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:18.252118 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:18.253286 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:18.255174 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:18.256075 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:18.256245 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:18.260583 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:18.260589 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:18.260593 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:18.260631 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:18.263518 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:18.264430 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:18.264434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:18.264436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:18.264566 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:18.265058 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:18.265062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:18.265064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:18.265364 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:18.265530 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:18.266330 (+ 800us) server_negotiation.cc:299] Negotiation successful
0619 15:01:18.266585 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1401,"thread_start_us":1255,"threads_started":1}
I20260619 15:01:18.268095 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:18.246445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45345 (local address 127.11.175.193:34115)
0619 15:01:18.247591 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:18.247673 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:18.249847 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:18.250619 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:18.250628 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:18.251322 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:18.251929 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:18.251949 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:18.253513 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:18.253517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:18.254988 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:18.255003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:18.256419 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:18.256425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:18.256658 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:18.257688 (+ 1030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:01:18.257714 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:18.260250 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:18.263732 (+ 3482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:18.263743 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:18.263763 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:18.264270 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:18.264687 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:18.264693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:18.264696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:18.264923 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:18.265512 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:18.265520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:18.265911 (+ 391us) client_negotiation.cc:769] Sending connection context
0619 15:01:18.266197 (+ 286us) client_negotiation.cc:240] Negotiation successful
0619 15:01:18.266839 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":422,"threads_started":1}
I20260619 15:01:18.269860 29333 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45345
I20260619 15:01:18.270229 29333 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:18.271109 29333 heartbeater.cc:511] Master 127.11.175.254:45345 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 15:01:18.273914 29144 ts_manager.cc:194] Registered new tserver with Master: 622e52f43d9a413b96e717d3888e0d7b (127.11.175.193:36587)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
I20260619 15:01:18.275820 29144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:34115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 15:01:18.306834 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45345
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:18.662808 29339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:18.663110 29339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:18.663270 29339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:18.670908 29339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:18.671036 29339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:18.671250 29339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 15:01:18.681826 29339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:18.683353 29339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:18.684823 29339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:18.699542 29344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:18.699461 29345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:18.699488 29347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:18.700670 29339 server_base.cc:1034] running on GCE node
I20260619 15:01:18.701565 29339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:18.702888 29339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:18.704331 29339 hybrid_clock.cc:648] HybridClock initialized: now 1781881278704291 us; error 44 us; skew 500 ppm
I20260619 15:01:18.704674 29339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:18 dist-test-slave-76rk krb5kdc[29099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881278, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:18.708300 29339 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 15:01:18.710173 29339 webserver.cc:460] Webserver started at http://127.11.175.194:35825/ using document root <none> and password file <none>
I20260619 15:01:18.711038 29339 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:18.711166 29339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:18.711449 29339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:18.714787 29339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "02352459ed844411bd7493ddfd583efb"
format_stamp: "Formatted at 2026-06-19 15:01:18 on dist-test-slave-76rk"
server_key: "ec9d8aa7666e1ff7408cd5448bdc8641"
server_key_iv: "bbcc7072f671b427967f2b5da833b888"
server_key_version: "encryptionkey@0"
I20260619 15:01:18.715600 29339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "02352459ed844411bd7493ddfd583efb"
format_stamp: "Formatted at 2026-06-19 15:01:18 on dist-test-slave-76rk"
server_key: "ec9d8aa7666e1ff7408cd5448bdc8641"
server_key_iv: "bbcc7072f671b427967f2b5da833b888"
server_key_version: "encryptionkey@0"
I20260619 15:01:18.721432 29339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 15:01:18.725502 29354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:18.727264 29339 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260619 15:01:18.727483 29339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "02352459ed844411bd7493ddfd583efb"
format_stamp: "Formatted at 2026-06-19 15:01:18 on dist-test-slave-76rk"
server_key: "ec9d8aa7666e1ff7408cd5448bdc8641"
server_key_iv: "bbcc7072f671b427967f2b5da833b888"
server_key_version: "encryptionkey@0"
I20260619 15:01:18.727734 29339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:18.741595 29339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:18.747210 29339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:18.747560 29339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:18.748687 29339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:18.750257 29339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:18.750371 29339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:18.750578 29339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:18.750690 29339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:18.789640 29339 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:36373
I20260619 15:01:18.789659 29466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:36373 every 8 connection(s)
I20260619 15:01:18.791690 29339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 15:01:18.795665 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 29339
I20260619 15:01:18.795893 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 15:01:18.796607 11967 external_mini_cluster.cc:1428] Setting key c6b7a08d4c4435dd6aa6ff6ea1f6ac6b
Jun 19 15:01:18 dist-test-slave-76rk krb5kdc[29099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881278, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:18.821930 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:18.794983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:45475 (local address 127.11.175.254:45345)
0619 15:01:18.795577 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:18.795584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:18.797160 (+ 1576us) server_negotiation.cc:407] Connection header received
0619 15:01:18.799991 (+ 2831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:18.799995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:18.800266 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:18.800374 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:18.803064 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:18.804057 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:18.807047 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:18.807546 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:18.807883 (+ 337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:18.812853 (+ 4970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:18.812863 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:18.812871 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:18.812925 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:18.816930 (+ 4005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:18.818160 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:18.818169 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:18.818175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:18.818364 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:18.819562 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:18.819570 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:18.819577 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:18.820049 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:18.820384 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:18.821117 (+ 733us) server_negotiation.cc:299] Negotiation successful
0619 15:01:18.821431 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":302,"threads_started":1}
I20260619 15:01:18.822628 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:18.795706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45345 (local address 127.11.175.194:45475)
0619 15:01:18.796908 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:18.796985 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:18.799586 (+ 2601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:18.800715 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:18.800731 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:18.801622 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:18.802802 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:18.802840 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:18.804254 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:18.804263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:18.806814 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:18.806838 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:18.807855 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:18.807864 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:18.808303 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:18.809212 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:01:18.809250 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:18.812509 (+ 3259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:18.817191 (+ 4682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:18.817200 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:18.817216 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:18.817957 (+ 741us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:18.818924 (+ 967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:18.818928 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:18.818931 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:18.819395 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:18.820233 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:18.820240 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:18.820682 (+ 442us) client_negotiation.cc:769] Sending connection context
0619 15:01:18.820946 (+ 264us) client_negotiation.cc:240] Negotiation successful
0619 15:01:18.821417 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":415,"threads_started":1}
I20260619 15:01:18.824487 29467 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45345
I20260619 15:01:18.824833 29467 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:18.825700 29467 heartbeater.cc:511] Master 127.11.175.254:45345 requested a full tablet report, sending...
I20260619 15:01:18.827435 29144 ts_manager.cc:194] Registered new tserver with Master: 02352459ed844411bd7493ddfd583efb (127.11.175.194:36373)
I20260619 15:01:18.828764 29144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:45475
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 15:01:18.863366 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45345
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:19.224637 29473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:19.224938 29473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:19.225106 29473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:19.233026 29473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:19.233160 29473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:19.233383 29473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 15:01:19.243918 29473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:19.245227 29473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:19.246747 29473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:19.260740 29478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:19.260810 29481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:19.260728 29479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:19.261518 29473 server_base.cc:1034] running on GCE node
I20260619 15:01:19.262213 29473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:19.263357 29473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:19.264600 29473 hybrid_clock.cc:648] HybridClock initialized: now 1781881279264552 us; error 39 us; skew 500 ppm
I20260619 15:01:19.264962 29473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:19 dist-test-slave-76rk krb5kdc[29099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881279, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:19.268324 29473 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 15:01:19.269905 29473 webserver.cc:460] Webserver started at http://127.11.175.195:46847/ using document root <none> and password file <none>
I20260619 15:01:19.270598 29473 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:19.270802 29473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:19.271116 29473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:19.274531 29473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "4e4e42c75e6c4619a0d620155b83218a"
format_stamp: "Formatted at 2026-06-19 15:01:19 on dist-test-slave-76rk"
server_key: "2e8b35a491ad6fb155f623353d0a525c"
server_key_iv: "2563e374387750dca9813440823a8117"
server_key_version: "encryptionkey@0"
I20260619 15:01:19.275403 29473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "4e4e42c75e6c4619a0d620155b83218a"
format_stamp: "Formatted at 2026-06-19 15:01:19 on dist-test-slave-76rk"
server_key: "2e8b35a491ad6fb155f623353d0a525c"
server_key_iv: "2563e374387750dca9813440823a8117"
server_key_version: "encryptionkey@0"
I20260619 15:01:19.279546 29333 heartbeater.cc:503] Master 127.11.175.254:45345 was elected leader, sending a full tablet report...
I20260619 15:01:19.281245 29473 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 15:01:19.285384 29488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:19.287124 29473 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:01:19.287381 29473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "4e4e42c75e6c4619a0d620155b83218a"
format_stamp: "Formatted at 2026-06-19 15:01:19 on dist-test-slave-76rk"
server_key: "2e8b35a491ad6fb155f623353d0a525c"
server_key_iv: "2563e374387750dca9813440823a8117"
server_key_version: "encryptionkey@0"
I20260619 15:01:19.287619 29473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:19.316315 29473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:19.321182 29473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:19.321517 29473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:19.322727 29473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:19.324258 29473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:19.324375 29473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:19.324527 29473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:19.324599 29473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:19.361250 29473 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:38065
I20260619 15:01:19.361302 29600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:38065 every 8 connection(s)
I20260619 15:01:19.362588 29473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 15:01:19.366411 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 29473
I20260619 15:01:19.366614 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 15:01:19.367170 11967 external_mini_cluster.cc:1428] Setting key 04a11f8ebb87459b7fdc091f17207876
Jun 19 15:01:19 dist-test-slave-76rk krb5kdc[29099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881279, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:19.384675 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.365323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:52843 (local address 127.11.175.254:45345)
0619 15:01:19.365797 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:19.365803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:19.366992 (+ 1189us) server_negotiation.cc:407] Connection header received
0619 15:01:19.369153 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:19.369158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:19.369245 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:19.369359 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:19.371357 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.372204 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.373902 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.374344 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.374450 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.378137 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:19.378143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:19.378147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:19.378185 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:19.381276 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:19.382193 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:19.382198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:19.382202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:19.382355 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:19.382906 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:19.382910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:19.382912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:19.383302 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:19.383485 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:19.384207 (+ 722us) server_negotiation.cc:299] Negotiation successful
0619 15:01:19.384397 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":221,"threads_started":1}
I20260619 15:01:19.385744 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.365910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45345 (local address 127.11.175.195:52843)
0619 15:01:19.366815 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:19.366861 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:19.368833 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:19.369654 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:19.369665 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:19.370526 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:19.371190 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.371209 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.372355 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.372358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.373697 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.373714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.374550 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.374555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.374840 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.375520 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:01:19.375550 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:19.377832 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:19.381457 (+ 3625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:19.381463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:19.381478 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:19.382047 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:19.382480 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:19.382485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:19.382489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:19.382777 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:19.383470 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:19.383477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:19.383837 (+ 360us) client_negotiation.cc:769] Sending connection context
0619 15:01:19.384080 (+ 243us) client_negotiation.cc:240] Negotiation successful
0619 15:01:19.384550 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":415,"threads_started":1}
I20260619 15:01:19.387331 29601 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45345
I20260619 15:01:19.387799 29601 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:19.388566 29601 heartbeater.cc:511] Master 127.11.175.254:45345 requested a full tablet report, sending...
I20260619 15:01:19.390120 29144 ts_manager.cc:194] Registered new tserver with Master: 4e4e42c75e6c4619a0d620155b83218a (127.11.175.195:38065)
I20260619 15:01:19.391103 29144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:52843
I20260619 15:01:19.396417 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 15:01:19.416407 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.401611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56608 (local address 127.11.175.254:45345)
0619 15:01:19.401852 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:19.401859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:19.402155 (+ 296us) server_negotiation.cc:407] Connection header received
0619 15:01:19.402275 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:19.402280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:19.402351 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:19.402458 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:19.404145 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.404940 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.406597 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.407110 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.407232 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.410027 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:19.410033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:19.410037 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:19.410075 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:19.413019 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:19.413987 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:19.413991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:19.413994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:19.414212 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:19.414801 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:19.414805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:19.414807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:19.415175 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:19.415378 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:19.415822 (+ 444us) server_negotiation.cc:299] Negotiation successful
0619 15:01:19.416033 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 15:01:19.421530 29144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 15:01:19.429277 29144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 15:01:19.454201 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.444477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.11.175.193:36587)
0619 15:01:19.446681 (+ 2204us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:19.446692 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:19.446726 (+ 34us) server_negotiation.cc:407] Connection header received
0619 15:01:19.446848 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:19.446857 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:19.447132 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:19.447360 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:19.448268 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.449517 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.451605 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.452950 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.453174 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.453373 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:19.453670 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 15:01:19.453844 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1994,"thread_start_us":1853,"threads_started":1}
I20260619 15:01:19.454232 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.443728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36587 (local address 127.0.0.1:43222)
0619 15:01:19.445649 (+ 1921us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:19.445707 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:19.445925 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:19.447360 (+ 1435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:19.447368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:19.447396 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:19.447755 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.447769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.449660 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.449669 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.451390 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.451407 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.453164 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.453170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.453390 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.453459 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 15:01:19.453616 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 15:01:19.453829 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":3009,"threads_started":1}
I20260619 15:01:19.463956 29268 tablet_service.cc:1467] Processing CreateTablet for tablet 266e83ed3df84277b78d7cbf3f280a20 (DEFAULT_TABLE table=test-table [id=d24ab6bdc27144a1935151648c912fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:19.465018 29614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.443946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38065 (local address 127.0.0.1:32788)
0619 15:01:19.447007 (+ 3061us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:19.447024 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:19.447138 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:19.457584 (+ 10446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:19.457588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:19.457614 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:19.457962 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.457970 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.459805 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.459810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.462215 (+ 2405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.462233 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.464109 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.464116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.464312 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.464351 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 15:01:19.464459 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 15:01:19.464613 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2945,"thread_start_us":709,"threads_started":1}
I20260619 15:01:19.465060 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.444266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.11.175.195:38065)
0619 15:01:19.456763 (+ 12497us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:19.456776 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:19.456837 (+ 61us) server_negotiation.cc:407] Connection header received
0619 15:01:19.456957 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:19.456969 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:19.457302 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:19.457578 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:19.458172 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.459632 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.462398 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.463881 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.464110 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.464305 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:19.464516 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 15:01:19.464702 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12263,"thread_start_us":15347,"threads_started":1}
I20260619 15:01:19.465502 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.447978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:36373 (local address 127.0.0.1:41826)
0619 15:01:19.454477 (+ 6499us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:19.454498 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:19.454599 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:19.459066 (+ 4467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:19.459072 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:19.459101 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:19.459349 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.459360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.461333 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.461339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.463162 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.463176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.464767 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.464774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.464955 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.464998 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 15:01:19.465074 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 15:01:19.465256 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6380,"thread_start_us":11478,"threads_started":1}
I20260619 15:01:19.465878 29268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 266e83ed3df84277b78d7cbf3f280a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:19.466243 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.447537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41826 (local address 127.11.175.194:36373)
0619 15:01:19.458259 (+ 10722us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:19.458271 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:19.458308 (+ 37us) server_negotiation.cc:407] Connection header received
0619 15:01:19.458439 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:19.458451 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:19.458806 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:19.459052 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:19.459527 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.461146 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.463346 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.464563 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.465450 (+ 887us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.465625 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:19.465729 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 15:01:19.465905 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10578,"thread_start_us":10458,"threads_started":1}
I20260619 15:01:19.469441 29536 tablet_service.cc:1467] Processing CreateTablet for tablet 266e83ed3df84277b78d7cbf3f280a20 (DEFAULT_TABLE table=test-table [id=d24ab6bdc27144a1935151648c912fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:19.469981 29402 tablet_service.cc:1467] Processing CreateTablet for tablet 266e83ed3df84277b78d7cbf3f280a20 (DEFAULT_TABLE table=test-table [id=d24ab6bdc27144a1935151648c912fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:19.470854 29536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 266e83ed3df84277b78d7cbf3f280a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:19.471419 29402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 266e83ed3df84277b78d7cbf3f280a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:19.482261 29620 tablet_bootstrap.cc:492] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: Bootstrap starting.
I20260619 15:01:19.486393 29622 tablet_bootstrap.cc:492] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: Bootstrap starting.
I20260619 15:01:19.488420 29620 tablet_bootstrap.cc:654] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:19.488659 29621 tablet_bootstrap.cc:492] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: Bootstrap starting.
I20260619 15:01:19.489759 29622 tablet_bootstrap.cc:654] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:19.490221 29620 log.cc:824] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:19.490945 29622 log.cc:824] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:19.492483 29621 tablet_bootstrap.cc:654] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:19.493834 29621 log.cc:824] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:19.493935 29620 tablet_bootstrap.cc:492] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: No bootstrap required, opened a new log
I20260619 15:01:19.494271 29620 ts_tablet_manager.cc:1397] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260619 15:01:19.497558 29622 tablet_bootstrap.cc:492] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: No bootstrap required, opened a new log
I20260619 15:01:19.497840 29621 tablet_bootstrap.cc:492] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: No bootstrap required, opened a new log
I20260619 15:01:19.497938 29622 ts_tablet_manager.cc:1397] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260619 15:01:19.498116 29621 ts_tablet_manager.cc:1397] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260619 15:01:19.505815 29622 raft_consensus.cc:348] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 } } peers { permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 } } peers { permanent_uuid: "02352459ed844411bd7493ddfd583efb" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 36373 } }
I20260619 15:01:19.505828 29620 raft_consensus.cc:348] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 } } peers { permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 } } peers { permanent_uuid: "02352459ed844411bd7493ddfd583efb" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 36373 } }
I20260619 15:01:19.506268 29622 raft_consensus.cc:374] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:19.506280 29620 raft_consensus.cc:374] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:19.506369 29622 raft_consensus.cc:729] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02352459ed844411bd7493ddfd583efb, State: Initialized, Role: FOLLOWER
I20260619 15:01:19.506376 29620 raft_consensus.cc:729] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622e52f43d9a413b96e717d3888e0d7b, State: Initialized, Role: FOLLOWER
I20260619 15:01:19.507064 29622 consensus_queue.cc:260] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 } } peers { permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 } } peers { permanent_uuid: "02352459ed844411bd7493ddfd583efb" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 36373 } }
I20260619 15:01:19.507074 29620 consensus_queue.cc:260] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 } } peers { permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 } } peers { permanent_uuid: "02352459ed844411bd7493ddfd583efb" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 36373 } }
I20260619 15:01:19.508973 29467 heartbeater.cc:503] Master 127.11.175.254:45345 was elected leader, sending a full tablet report...
I20260619 15:01:19.509160 29622 ts_tablet_manager.cc:1428] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260619 15:01:19.510821 29621 raft_consensus.cc:348] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 } } peers { permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 } } peers { permanent_uuid: "02352459ed844411bd7493ddfd583efb" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 36373 } }
I20260619 15:01:19.511322 29621 raft_consensus.cc:374] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:19.511413 29621 raft_consensus.cc:729] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e4e42c75e6c4619a0d620155b83218a, State: Initialized, Role: FOLLOWER
I20260619 15:01:19.512271 29620 ts_tablet_manager.cc:1428] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: Time spent starting tablet: real 0.018s user 0.006s sys 0.010s
I20260619 15:01:19.512275 29621 consensus_queue.cc:260] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 } } peers { permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 } } peers { permanent_uuid: "02352459ed844411bd7493ddfd583efb" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 36373 } }
I20260619 15:01:19.513752 29601 heartbeater.cc:503] Master 127.11.175.254:45345 was elected leader, sending a full tablet report...
I20260619 15:01:19.515676 29621 ts_tablet_manager.cc:1428] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
W20260619 15:01:19.544044 29468 tablet.cc:2369] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:01:19.567795 29626 raft_consensus.cc:482] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 15:01:19.568082 29626 raft_consensus.cc:504] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 } } peers { permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 } } peers { permanent_uuid: "02352459ed844411bd7493ddfd583efb" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 36373 } }
I20260619 15:01:19.570268 29626 leader_election.cc:288] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 622e52f43d9a413b96e717d3888e0d7b (127.11.175.193:36587), 4e4e42c75e6c4619a0d620155b83218a (127.11.175.195:38065)
I20260619 15:01:19.577627 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.571112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:50993 (local address 127.11.175.195:38065)
0619 15:01:19.571384 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:19.571393 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:19.571771 (+ 378us) server_negotiation.cc:407] Connection header received
0619 15:01:19.571984 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:19.571988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:19.572049 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:19.572172 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:19.572814 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.573697 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.575796 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.576629 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.576848 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.576933 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:19.577231 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 15:01:19.577364 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 15:01:19.577719 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.570754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:44249 (local address 127.11.175.193:36587)
0619 15:01:19.571004 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:19.571013 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:19.572068 (+ 1055us) server_negotiation.cc:407] Connection header received
0619 15:01:19.572270 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:19.572274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:19.572331 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:19.572427 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:19.573135 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.573910 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.575884 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.576724 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.576930 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.577009 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:19.577313 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 15:01:19.577452 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 15:01:19.577831 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.570620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:38065 (local address 127.11.175.194:50993)
0619 15:01:19.571618 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:19.571647 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:19.571840 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:19.572204 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:19.572209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:19.572240 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:19.572656 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.572667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.573832 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.573840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.575610 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.575632 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.576880 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.576887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.577056 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.577085 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 15:01:19.577200 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 15:01:19.577347 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":6,"thread_start_us":564,"threads_started":1}
I20260619 15:01:19.577869 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.570798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:36587 (local address 127.11.175.194:44249)
0619 15:01:19.571953 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:19.571969 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:19.572120 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:19.572442 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:19.572446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:19.572473 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:19.572959 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.572976 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.574072 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.574077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.575705 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:19.575718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.576960 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.576967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:19.577152 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.577176 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 15:01:19.577283 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 15:01:19.577430 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":21,"thread_start_us":815,"threads_started":1}
I20260619 15:01:19.579077 29556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266e83ed3df84277b78d7cbf3f280a20" candidate_uuid: "02352459ed844411bd7493ddfd583efb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e4e42c75e6c4619a0d620155b83218a" is_pre_election: true
I20260619 15:01:19.579080 29288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266e83ed3df84277b78d7cbf3f280a20" candidate_uuid: "02352459ed844411bd7493ddfd583efb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622e52f43d9a413b96e717d3888e0d7b" is_pre_election: true
I20260619 15:01:19.579599 29556 raft_consensus.cc:2454] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02352459ed844411bd7493ddfd583efb in term 0.
I20260619 15:01:19.579599 29288 raft_consensus.cc:2454] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02352459ed844411bd7493ddfd583efb in term 0.
I20260619 15:01:19.580255 29357 leader_election.cc:302] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02352459ed844411bd7493ddfd583efb, 4e4e42c75e6c4619a0d620155b83218a; no voters:
I20260619 15:01:19.580709 29626 raft_consensus.cc:2781] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 15:01:19.580842 29626 raft_consensus.cc:482] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 15:01:19.580915 29626 raft_consensus.cc:3037] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:19.583240 29626 raft_consensus.cc:504] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 } } peers { permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 } } peers { permanent_uuid: "02352459ed844411bd7493ddfd583efb" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 36373 } }
I20260619 15:01:19.583925 29626 leader_election.cc:288] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [CANDIDATE]: Term 1 election: Requested vote from peers 622e52f43d9a413b96e717d3888e0d7b (127.11.175.193:36587), 4e4e42c75e6c4619a0d620155b83218a (127.11.175.195:38065)
I20260619 15:01:19.584453 29288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266e83ed3df84277b78d7cbf3f280a20" candidate_uuid: "02352459ed844411bd7493ddfd583efb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622e52f43d9a413b96e717d3888e0d7b"
I20260619 15:01:19.584596 29556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266e83ed3df84277b78d7cbf3f280a20" candidate_uuid: "02352459ed844411bd7493ddfd583efb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e4e42c75e6c4619a0d620155b83218a"
I20260619 15:01:19.584687 29288 raft_consensus.cc:3037] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:19.584870 29556 raft_consensus.cc:3037] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:19.587105 29288 raft_consensus.cc:2454] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02352459ed844411bd7493ddfd583efb in term 1.
I20260619 15:01:19.587411 29556 raft_consensus.cc:2454] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02352459ed844411bd7493ddfd583efb in term 1.
I20260619 15:01:19.587708 29355 leader_election.cc:302] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02352459ed844411bd7493ddfd583efb, 622e52f43d9a413b96e717d3888e0d7b; no voters:
I20260619 15:01:19.588059 29626 raft_consensus.cc:2781] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:19.588553 29626 raft_consensus.cc:686] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 1 LEADER]: Becoming Leader. State: Replica: 02352459ed844411bd7493ddfd583efb, State: Running, Role: LEADER
I20260619 15:01:19.589113 29626 consensus_queue.cc:237] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 } } peers { permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 } } peers { permanent_uuid: "02352459ed844411bd7493ddfd583efb" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 36373 } }
I20260619 15:01:19.594062 29142 catalog_manager.cc:5495] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb reported cstate change: term changed from 0 to 1, leader changed from <none> to 02352459ed844411bd7493ddfd583efb (127.11.175.194). New cstate: current_term: 1 leader_uuid: "02352459ed844411bd7493ddfd583efb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02352459ed844411bd7493ddfd583efb" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 36373 } health_report { overall_health: HEALTHY } } }
I20260619 15:01:19.613220 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.607696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.11.175.194:36373)
0619 15:01:19.607917 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:19.607922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:19.607940 (+ 18us) server_negotiation.cc:407] Connection header received
0619 15:01:19.608031 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:19.608035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:19.608172 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:19.608306 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:01:19.608728 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.609434 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.610918 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.611387 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.611489 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.611994 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:01:19.612549 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:01:19.612678 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:19.612785 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 15:01:19.612856 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260619 15:01:19.614367 29602 tablet.cc:2369] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:01:19.622700 29556 raft_consensus.cc:1261] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a [term 1 FOLLOWER]: Refusing update from remote peer 02352459ed844411bd7493ddfd583efb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:19.622707 29288 raft_consensus.cc:1261] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b [term 1 FOLLOWER]: Refusing update from remote peer 02352459ed844411bd7493ddfd583efb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:19.623878 29626 consensus_queue.cc:1035] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "622e52f43d9a413b96e717d3888e0d7b" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 36587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:19.624267 29631 consensus_queue.cc:1035] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e4e42c75e6c4619a0d620155b83218a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 38065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:19.635903 29634 mvcc.cc:205] Tried to move back new op lower bound from 7298585721323778048 to 7298585721201111040. Current Snapshot: MvccSnapshot[applied={T|T < 7298585721323778048}]
I20260619 15:01:19.655977 29142 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:19.656280 29142 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:19.660254 29142 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6cb38a94ef244db9af3c13a7d39c72f: Sending DeleteTablet for 3 replicas of tablet 266e83ed3df84277b78d7cbf3f280a20
I20260619 15:01:19.661690 29268 tablet_service.cc:1514] Processing DeleteTablet for tablet 266e83ed3df84277b78d7cbf3f280a20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:19 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:43222
I20260619 15:01:19.661648 29536 tablet_service.cc:1514] Processing DeleteTablet for tablet 266e83ed3df84277b78d7cbf3f280a20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:19 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:32788
I20260619 15:01:19.661914 29402 tablet_service.cc:1514] Processing DeleteTablet for tablet 266e83ed3df84277b78d7cbf3f280a20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:19 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:41826
I20260619 15:01:19.664350 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 29205
I20260619 15:01:19.678742 29645 tablet_replica.cc:330] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: stopping tablet replica
I20260619 15:01:19.678857 29644 tablet_replica.cc:330] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: stopping tablet replica
I20260619 15:01:19.679273 29644 raft_consensus.cc:2229] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:19.679275 29645 raft_consensus.cc:2229] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:19.679864 29644 raft_consensus.cc:2258] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:19.679864 29645 raft_consensus.cc:2258] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:19.680676 29646 tablet_replica.cc:330] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: stopping tablet replica
I20260619 15:01:19.681123 29646 raft_consensus.cc:2229] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 1 LEADER]: Raft consensus shutting down.
I20260619 15:01:19.681880 29646 raft_consensus.cc:2258] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:19.682844 29644 ts_tablet_manager.cc:1905] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:19.684000 29645 ts_tablet_manager.cc:1905] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:19.688126 29646 ts_tablet_manager.cc:1905] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 15:01:19 dist-test-slave-76rk krb5kdc[29099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 15:01:19.693102 29645 ts_tablet_manager.cc:1918] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:19.693324 29645 log.cc:1196] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/266e83ed3df84277b78d7cbf3f280a20
I20260619 15:01:19.693977 29645 ts_tablet_manager.cc:1939] T 266e83ed3df84277b78d7cbf3f280a20 P 622e52f43d9a413b96e717d3888e0d7b: Deleting consensus metadata
I20260619 15:01:19.696782 29128 catalog_manager.cc:4841] TS 622e52f43d9a413b96e717d3888e0d7b (127.11.175.193:36587): tablet 266e83ed3df84277b78d7cbf3f280a20 (table test-table [id=d24ab6bdc27144a1935151648c912fca]) successfully deleted
I20260619 15:01:19.698143 29644 ts_tablet_manager.cc:1918] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:19.698356 29644 log.cc:1196] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/266e83ed3df84277b78d7cbf3f280a20
I20260619 15:01:19.698973 29644 ts_tablet_manager.cc:1939] T 266e83ed3df84277b78d7cbf3f280a20 P 4e4e42c75e6c4619a0d620155b83218a: Deleting consensus metadata
I20260619 15:01:19.700364 29646 ts_tablet_manager.cc:1918] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:19.700546 29646 log.cc:1196] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/266e83ed3df84277b78d7cbf3f280a20
I20260619 15:01:19.701007 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.666304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.11.175.193:36587)
0619 15:01:19.666545 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:19.666554 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:19.673562 (+ 7008us) server_negotiation.cc:407] Connection header received
0619 15:01:19.674806 (+ 1244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:19.674815 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:19.674950 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:19.675047 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:19.680333 (+ 5286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.681524 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.683624 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.684296 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.684403 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.690579 (+ 6176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:19.690598 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:19.690617 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:19.690704 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:19.695226 (+ 4522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:19.696326 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:19.696336 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:19.696340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:19.696536 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:19.698460 (+ 1924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:19.698468 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:19.698474 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:19.699027 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:19.699342 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:19.700139 (+ 797us) server_negotiation.cc:299] Negotiation successful
0619 15:01:19.700414 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 15:01:19.701081 29130 catalog_manager.cc:4841] TS 4e4e42c75e6c4619a0d620155b83218a (127.11.175.195:38065): tablet 266e83ed3df84277b78d7cbf3f280a20 (table test-table [id=d24ab6bdc27144a1935151648c912fca]) successfully deleted
I20260619 15:01:19.701735 29646 ts_tablet_manager.cc:1939] T 266e83ed3df84277b78d7cbf3f280a20 P 02352459ed844411bd7493ddfd583efb: Deleting consensus metadata
I20260619 15:01:19.702315 29328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:19.703794 29128 catalog_manager.cc:4841] TS 02352459ed844411bd7493ddfd583efb (127.11.175.194:36373): tablet 266e83ed3df84277b78d7cbf3f280a20 (table test-table [id=d24ab6bdc27144a1935151648c912fca]) successfully deleted
I20260619 15:01:19.831265 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 29205
I20260619 15:01:19.850034 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 29339
Jun 19 15:01:19 dist-test-slave-76rk krb5kdc[29099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 15:01:19.869985 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:19.850943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41842 (local address 127.11.175.194:36373)
0619 15:01:19.851219 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:19.851225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:19.851246 (+ 21us) server_negotiation.cc:407] Connection header received
0619 15:01:19.851378 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:19.851383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:19.851489 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:19.851604 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:19.853253 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.854085 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.855671 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:19.856380 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:19.856546 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:19.861521 (+ 4975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:19.861543 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:19.861565 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:19.861643 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:19.866562 (+ 4919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:19.867504 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:19.867515 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:19.867519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:19.867720 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:19.868176 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:19.868182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:19.868186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:19.868704 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:19.869081 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:19.869319 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 15:01:19.869573 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 15:01:19.871111 29462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:19.990537 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 29339
I20260619 15:01:20.013332 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 29473
Jun 19 15:01:20 dist-test-slave-76rk krb5kdc[29099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 15:01:20.035461 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:20.014346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32792 (local address 127.11.175.195:38065)
0619 15:01:20.014677 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:20.014685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:20.014753 (+ 68us) server_negotiation.cc:407] Connection header received
0619 15:01:20.015097 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:20.015101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:20.015186 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:20.015310 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:20.016817 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:20.018035 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:20.019550 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:20.019980 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:20.020137 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:20.026997 (+ 6860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:20.027016 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:20.027031 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:20.027079 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:20.030776 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:20.032157 (+ 1381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:20.032167 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:20.032169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:20.032408 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:20.033080 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:20.033086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:20.033089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:20.033907 (+ 818us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:20.034203 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:20.034816 (+ 613us) server_negotiation.cc:299] Negotiation successful
0619 15:01:20.035110 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260619 15:01:20.036152 29596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:20.130790 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 29473
I20260619 15:01:20.147894 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 29112
I20260619 15:01:20.149087 29174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:20.261453 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 29112
2026-06-19T15:01:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4612 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T15:01:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:20Z Disabled control of system clock
I20260619 15:01:20.344544 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:33781
--webserver_interface=127.11.175.254
--webserver_port=0
--builtin_ntp_servers=127.11.175.212:43695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:33781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 15:01:20.689046 29655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:20.689354 29655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:20.689493 29655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:20.697096 29655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:20.697227 29655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:20.697304 29655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:20.697412 29655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:20.697512 29655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:20.707086 29655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:33781
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:33781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:20.708601 29655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:20.710482 29655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:20.723609 29661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:20.723875 29663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:20.723656 29660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:20.724557 29655 server_base.cc:1034] running on GCE node
I20260619 15:01:20.725407 29655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:20.726933 29655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:20.728170 29655 hybrid_clock.cc:648] HybridClock initialized: now 1781881280728121 us; error 51 us; skew 500 ppm
I20260619 15:01:20.728556 29655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:20.731141 29655 webserver.cc:460] Webserver started at http://127.11.175.254:33501/ using document root <none> and password file <none>
I20260619 15:01:20.731886 29655 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:20.732028 29655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:20.732376 29655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:20.735555 29655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "6fc748f694644ff9976df0c5af60d412"
format_stamp: "Formatted at 2026-06-19 15:01:20 on dist-test-slave-76rk"
server_key: "527668b99a44d3eaa8462d8bdb2362c7"
server_key_iv: "babe47e3d24224ca5e3fdd1b29390be8"
server_key_version: "encryptionkey@0"
I20260619 15:01:20.736425 29655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "6fc748f694644ff9976df0c5af60d412"
format_stamp: "Formatted at 2026-06-19 15:01:20 on dist-test-slave-76rk"
server_key: "527668b99a44d3eaa8462d8bdb2362c7"
server_key_iv: "babe47e3d24224ca5e3fdd1b29390be8"
server_key_version: "encryptionkey@0"
I20260619 15:01:20.742048 29655 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 15:01:20.746114 29669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:20.747828 29655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 15:01:20.747985 29655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "6fc748f694644ff9976df0c5af60d412"
format_stamp: "Formatted at 2026-06-19 15:01:20 on dist-test-slave-76rk"
server_key: "527668b99a44d3eaa8462d8bdb2362c7"
server_key_iv: "babe47e3d24224ca5e3fdd1b29390be8"
server_key_version: "encryptionkey@0"
I20260619 15:01:20.748179 29655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:20.764778 29655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:20.765635 29655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:20.765885 29655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:20.786115 29655 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:33781
I20260619 15:01:20.786128 29720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:33781 every 8 connection(s)
I20260619 15:01:20.787599 29655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:20.791443 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 29655
I20260619 15:01:20.791632 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:20.791620 29721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:20.792104 11967 external_mini_cluster.cc:1428] Setting key 785c4293b06ef9c0826c07a1f10948ed
I20260619 15:01:20.804629 29721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412: Bootstrap starting.
I20260619 15:01:20.805846 29724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:20.794125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50138 (local address 127.11.175.254:33781)
0619 15:01:20.795259 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:20.795278 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:20.795328 (+ 50us) server_negotiation.cc:407] Connection header received
0619 15:01:20.797000 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:20.797022 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:20.797500 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:20.797914 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:20.798538 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:20.799887 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:20.801318 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:20.802021 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:20.802240 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:20.803497 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:20.803504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:20.803518 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:20.804048 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:20.804122 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:20.804142 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:20.804289 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:20.804471 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 15:01:20.804822 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":432,"threads_started":1}
I20260619 15:01:20.808171 29721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:20.809324 29721 log.cc:824] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:20.812224 29721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412: No bootstrap required, opened a new log
I20260619 15:01:20.819674 29721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc748f694644ff9976df0c5af60d412" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33781 } }
I20260619 15:01:20.820128 29721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:20.820247 29721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc748f694644ff9976df0c5af60d412, State: Initialized, Role: FOLLOWER
I20260619 15:01:20.820849 29721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc748f694644ff9976df0c5af60d412" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33781 } }
I20260619 15:01:20.821087 29721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:20.821220 29721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:20.821401 29721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:20.823698 29721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc748f694644ff9976df0c5af60d412" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33781 } }
I20260619 15:01:20.824246 29721 leader_election.cc:302] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fc748f694644ff9976df0c5af60d412; no voters:
I20260619 15:01:20.824877 29721 leader_election.cc:288] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:20.825053 29726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:20.825800 29726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [term 1 LEADER]: Becoming Leader. State: Replica: 6fc748f694644ff9976df0c5af60d412, State: Running, Role: LEADER
I20260619 15:01:20.826300 29726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc748f694644ff9976df0c5af60d412" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33781 } }
I20260619 15:01:20.827139 29721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:20.829195 29727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fc748f694644ff9976df0c5af60d412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc748f694644ff9976df0c5af60d412" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33781 } } }
I20260619 15:01:20.829676 29727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:20.830173 29728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fc748f694644ff9976df0c5af60d412. Latest consensus state: current_term: 1 leader_uuid: "6fc748f694644ff9976df0c5af60d412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc748f694644ff9976df0c5af60d412" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 33781 } } }
I20260619 15:01:20.830519 29728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:20.831815 29735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:20.837206 29735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:20.846560 29735 catalog_manager.cc:1347] Generated new cluster ID: 071f0bd04f9942dcb54ff1b75429ab9e
I20260619 15:01:20.846731 29735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:20.874133 29735 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:20.875604 29735 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:20.882331 29735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412: Generated new TSK 0
I20260619 15:01:20.883291 29735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:20.899209 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:33781
--builtin_ntp_servers=127.11.175.212:43695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 15:01:21.259831 29745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:21.260134 29745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:21.260282 29745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:21.267843 29745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:21.267961 29745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:21.268147 29745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 15:01:21.277673 29745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:33781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:21.279073 29745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:21.280937 29745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:21.295410 29753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:21.295444 29751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:21.295663 29750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:21.297128 29745 server_base.cc:1034] running on GCE node
I20260619 15:01:21.297873 29745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:21.299089 29745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:21.300458 29745 hybrid_clock.cc:648] HybridClock initialized: now 1781881281300380 us; error 71 us; skew 500 ppm
I20260619 15:01:21.300896 29745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:21.303674 29745 webserver.cc:460] Webserver started at http://127.11.175.193:33615/ using document root <none> and password file <none>
I20260619 15:01:21.304529 29745 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:21.304692 29745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:21.305209 29745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:21.308660 29745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "1bff859ac2494d1887044d19d26d8077"
format_stamp: "Formatted at 2026-06-19 15:01:21 on dist-test-slave-76rk"
server_key: "32c504ac46084f8610bdd4a2b8431fff"
server_key_iv: "54071fe2fa108931287713ba2372cd97"
server_key_version: "encryptionkey@0"
I20260619 15:01:21.309512 29745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "1bff859ac2494d1887044d19d26d8077"
format_stamp: "Formatted at 2026-06-19 15:01:21 on dist-test-slave-76rk"
server_key: "32c504ac46084f8610bdd4a2b8431fff"
server_key_iv: "54071fe2fa108931287713ba2372cd97"
server_key_version: "encryptionkey@0"
I20260619 15:01:21.315582 29745 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 15:01:21.319557 29759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:21.321034 29745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 15:01:21.321182 29745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "1bff859ac2494d1887044d19d26d8077"
format_stamp: "Formatted at 2026-06-19 15:01:21 on dist-test-slave-76rk"
server_key: "32c504ac46084f8610bdd4a2b8431fff"
server_key_iv: "54071fe2fa108931287713ba2372cd97"
server_key_version: "encryptionkey@0"
I20260619 15:01:21.321450 29745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:21.335361 29745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:21.336225 29745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:21.336542 29745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:21.337529 29745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:21.339078 29745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:21.339188 29745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:21.339329 29745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:21.339424 29745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:21.374965 29745 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:39279
I20260619 15:01:21.375020 29871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:39279 every 8 connection(s)
I20260619 15:01:21.376417 29745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 15:01:21.377813 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 29745
I20260619 15:01:21.378010 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 15:01:21.378526 11967 external_mini_cluster.cc:1428] Setting key 18ef2e866c2265ac3a97fe88926935d5
I20260619 15:01:21.381651 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:33781
--builtin_ntp_servers=127.11.175.212:43695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 15:01:21.399468 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:21.379624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:56997 (local address 127.11.175.254:33781)
0619 15:01:21.391165 (+ 11541us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:21.391170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:21.391187 (+ 17us) server_negotiation.cc:407] Connection header received
0619 15:01:21.391249 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:21.391253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:21.391322 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:21.391416 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:21.392752 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:21.394010 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:21.395907 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:21.396669 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:21.396844 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:21.398364 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:21.398373 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:21.398381 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:21.398410 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:21.398465 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:21.398487 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:21.398587 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:21.398897 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 15:01:21.399020 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11423,"thread_start_us":11325,"threads_started":1}
I20260619 15:01:21.400480 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:21.380660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:33781 (local address 127.11.175.193:56997)
0619 15:01:21.387678 (+ 7018us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:21.387751 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:21.390423 (+ 2672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:21.391791 (+ 1368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:21.391803 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:21.391911 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:21.392546 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:21.392564 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:21.394251 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:21.394256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:21.395726 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:21.395743 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:21.396916 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:21.396921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:21.397175 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:21.397969 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 15:01:21.397972 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:21.398017 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 15:01:21.398106 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:21.398601 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:21.398611 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:21.398618 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 15:01:21.398857 (+ 239us) client_negotiation.cc:240] Negotiation successful
0619 15:01:21.399258 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6303,"thread_start_us":6037,"threads_started":1}
I20260619 15:01:21.402321 29872 heartbeater.cc:347] Connected to a master server at 127.11.175.254:33781
I20260619 15:01:21.402698 29872 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:21.403614 29872 heartbeater.cc:511] Master 127.11.175.254:33781 requested a full tablet report, sending...
I20260619 15:01:21.405970 29686 ts_manager.cc:194] Registered new tserver with Master: 1bff859ac2494d1887044d19d26d8077 (127.11.175.193:39279)
I20260619 15:01:21.407582 29686 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.193:56997
W20260619 15:01:21.733918 29876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:21.734213 29876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:21.734362 29876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:21.741886 29876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:21.741995 29876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:21.742172 29876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 15:01:21.752198 29876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:33781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:21.753545 29876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:21.755371 29876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:21.769140 29881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:21.769270 29884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:21.769160 29882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:21.770174 29876 server_base.cc:1034] running on GCE node
I20260619 15:01:21.771006 29876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:21.772073 29876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:21.773319 29876 hybrid_clock.cc:648] HybridClock initialized: now 1781881281773262 us; error 56 us; skew 500 ppm
I20260619 15:01:21.773662 29876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:21.776266 29876 webserver.cc:460] Webserver started at http://127.11.175.194:45819/ using document root <none> and password file <none>
I20260619 15:01:21.777030 29876 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:21.777123 29876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:21.777385 29876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:21.780510 29876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "284ef330b6c249d6a3945fa870dc0c8a"
format_stamp: "Formatted at 2026-06-19 15:01:21 on dist-test-slave-76rk"
server_key: "f9628e21b22edd921bd22f14dbd80ee8"
server_key_iv: "d11b2c0068c0fa39815575e1124aab33"
server_key_version: "encryptionkey@0"
I20260619 15:01:21.781387 29876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "284ef330b6c249d6a3945fa870dc0c8a"
format_stamp: "Formatted at 2026-06-19 15:01:21 on dist-test-slave-76rk"
server_key: "f9628e21b22edd921bd22f14dbd80ee8"
server_key_iv: "d11b2c0068c0fa39815575e1124aab33"
server_key_version: "encryptionkey@0"
I20260619 15:01:21.787454 29876 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260619 15:01:21.792173 29890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:21.793960 29876 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260619 15:01:21.794147 29876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "284ef330b6c249d6a3945fa870dc0c8a"
format_stamp: "Formatted at 2026-06-19 15:01:21 on dist-test-slave-76rk"
server_key: "f9628e21b22edd921bd22f14dbd80ee8"
server_key_iv: "d11b2c0068c0fa39815575e1124aab33"
server_key_version: "encryptionkey@0"
I20260619 15:01:21.794394 29876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:21.820930 29876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:21.821841 29876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:21.822124 29876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:21.823354 29876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:21.825182 29876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:21.825393 29876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:21.825517 29876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:21.825662 29876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260619 15:01:21.862069 29876 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:43149
I20260619 15:01:21.862193 30002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:43149 every 8 connection(s)
I20260619 15:01:21.863523 29876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 15:01:21.873276 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 29876
I20260619 15:01:21.873503 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 15:01:21.874115 11967 external_mini_cluster.cc:1428] Setting key d348a40b9804f7b831f8053ef1f224c2
I20260619 15:01:21.877657 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:21.866279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:38597 (local address 127.11.175.254:33781)
0619 15:01:21.866492 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:21.866498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:21.867993 (+ 1495us) server_negotiation.cc:407] Connection header received
0619 15:01:21.869788 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:21.869793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:21.869870 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:21.869966 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:21.871410 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:21.872252 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:21.874030 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:21.874727 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:21.874885 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:21.876857 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:21.876862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:21.876866 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:21.876885 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:21.876920 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:21.876933 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:21.877017 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:21.877314 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 15:01:21.877403 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 15:01:21.877755 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:33781
--builtin_ntp_servers=127.11.175.212:43695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 15:01:21.879375 30005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:21.866858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:33781 (local address 127.11.175.194:38597)
0619 15:01:21.867845 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:21.867881 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:21.869490 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:21.870388 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:21.870398 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:21.870501 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:21.871211 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:21.871230 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:21.872414 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:21.872418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:21.873846 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:21.873859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:21.875191 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:21.875201 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:21.875522 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:21.876403 (+ 881us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 15:01:21.876405 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:21.876450 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 15:01:21.876584 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:21.877047 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:21.877059 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:21.877068 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 15:01:21.877286 (+ 218us) client_negotiation.cc:240] Negotiation successful
0619 15:01:21.877807 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":334,"threads_started":1}
I20260619 15:01:21.881073 30003 heartbeater.cc:347] Connected to a master server at 127.11.175.254:33781
I20260619 15:01:21.881484 30003 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:21.882323 30003 heartbeater.cc:511] Master 127.11.175.254:33781 requested a full tablet report, sending...
I20260619 15:01:21.884070 29686 ts_manager.cc:194] Registered new tserver with Master: 284ef330b6c249d6a3945fa870dc0c8a (127.11.175.194:43149)
I20260619 15:01:21.885514 29686 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.194:38597
W20260619 15:01:22.243352 30006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:22.243652 30006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:22.243742 30006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:22.251895 30006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:22.252066 30006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:22.252190 30006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 15:01:22.261799 30006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:33781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:22.263141 30006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:22.264938 30006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:22.278862 30011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:22.278820 30014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:22.278913 30012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:22.280086 30006 server_base.cc:1034] running on GCE node
I20260619 15:01:22.280824 30006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:22.281898 30006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:22.283144 30006 hybrid_clock.cc:648] HybridClock initialized: now 1781881282283083 us; error 75 us; skew 500 ppm
I20260619 15:01:22.283493 30006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:22.286053 30006 webserver.cc:460] Webserver started at http://127.11.175.195:33451/ using document root <none> and password file <none>
I20260619 15:01:22.286913 30006 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:22.287047 30006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:22.287381 30006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:22.290354 30006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "8ec040e6e0714d67ac8951d8d634d836"
format_stamp: "Formatted at 2026-06-19 15:01:22 on dist-test-slave-76rk"
server_key: "e78682440adf6c76480d50aa6cba9ee6"
server_key_iv: "1cbf64a559a949fda1209fdb48598377"
server_key_version: "encryptionkey@0"
I20260619 15:01:22.291296 30006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "8ec040e6e0714d67ac8951d8d634d836"
format_stamp: "Formatted at 2026-06-19 15:01:22 on dist-test-slave-76rk"
server_key: "e78682440adf6c76480d50aa6cba9ee6"
server_key_iv: "1cbf64a559a949fda1209fdb48598377"
server_key_version: "encryptionkey@0"
I20260619 15:01:22.296669 30006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 15:01:22.300772 30020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:22.302320 30006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 15:01:22.302527 30006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "8ec040e6e0714d67ac8951d8d634d836"
format_stamp: "Formatted at 2026-06-19 15:01:22 on dist-test-slave-76rk"
server_key: "e78682440adf6c76480d50aa6cba9ee6"
server_key_iv: "1cbf64a559a949fda1209fdb48598377"
server_key_version: "encryptionkey@0"
I20260619 15:01:22.302847 30006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:22.318300 30006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:22.319232 30006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:22.319509 30006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:22.320456 30006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:22.321947 30006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:22.322036 30006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:22.322113 30006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:22.322167 30006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:22.357318 30006 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:33931
I20260619 15:01:22.357414 30132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:33931 every 8 connection(s)
I20260619 15:01:22.358826 30006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 15:01:22.366528 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 30006
I20260619 15:01:22.366871 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 15:01:22.367539 11967 external_mini_cluster.cc:1428] Setting key cdaca86e20f5465c62277a804690b4cc
I20260619 15:01:22.373054 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.361377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:44829 (local address 127.11.175.254:33781)
0619 15:01:22.361608 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.361615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.363096 (+ 1481us) server_negotiation.cc:407] Connection header received
0619 15:01:22.365307 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.365312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.365390 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.365481 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:22.366813 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.368009 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.369958 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.370536 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.370702 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.372180 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:22.372185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:22.372188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:22.372207 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:22.372239 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:22.372251 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:22.372352 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.372594 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.372701 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 15:01:22.374197 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.361936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:33781 (local address 127.11.175.195:44829)
0619 15:01:22.362925 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:22.362969 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:22.364910 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:22.365759 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:22.365770 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:22.365873 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:22.366515 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.366534 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.368194 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.368202 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.369817 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.369830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.370836 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.370845 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.371094 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.371833 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 15:01:22.371836 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:22.371882 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 15:01:22.371970 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:22.372364 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:22.372377 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:22.372385 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 15:01:22.372568 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 15:01:22.373034 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":289,"threads_started":1}
I20260619 15:01:22.375730 30133 heartbeater.cc:347] Connected to a master server at 127.11.175.254:33781
I20260619 15:01:22.376071 30133 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:22.376884 30133 heartbeater.cc:511] Master 127.11.175.254:33781 requested a full tablet report, sending...
I20260619 15:01:22.378450 29686 ts_manager.cc:194] Registered new tserver with Master: 8ec040e6e0714d67ac8951d8d634d836 (127.11.175.195:33931)
I20260619 15:01:22.379581 29686 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.195:44829
I20260619 15:01:22.383208 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 15:01:22.393465 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.387330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37010 (local address 127.11.175.254:33781)
0619 15:01:22.387534 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.387539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.388145 (+ 606us) server_negotiation.cc:407] Connection header received
0619 15:01:22.388275 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.388279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.388346 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.388439 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:22.388993 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.389751 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.391197 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.391638 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.391741 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.392852 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:22.392856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:22.392859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:22.392876 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:22.392903 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:22.392913 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:22.392983 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.393156 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.393234 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 15:01:22.397745 29686 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 15:01:22.405090 29686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 15:01:22.411108 29872 heartbeater.cc:503] Master 127.11.175.254:33781 was elected leader, sending a full tablet report...
I20260619 15:01:22.450047 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.419857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39279 (local address 127.0.0.1:59450)
0619 15:01:22.433582 (+ 13725us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:22.433643 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:22.433879 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:22.436349 (+ 2470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:22.436361 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:22.436411 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:22.437010 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.437029 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.440167 (+ 3138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.440176 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.443800 (+ 3624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.443827 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.448928 (+ 5101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.448941 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.449111 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.449238 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 15:01:22.449419 (+ 181us) client_negotiation.cc:240] Negotiation successful
0619 15:01:22.449650 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13413,"mutex_wait_us":36,"thread_start_us":16341,"threads_started":1}
I20260619 15:01:22.450102 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.419994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59450 (local address 127.11.175.193:39279)
0619 15:01:22.435147 (+ 15153us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.435160 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.435202 (+ 42us) server_negotiation.cc:407] Connection header received
0619 15:01:22.435338 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.435351 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.435738 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.437421 (+ 1683us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:22.437771 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.439941 (+ 2170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.446839 (+ 6898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.448471 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.448824 (+ 353us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.449013 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.449505 (+ 492us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.449714 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14909,"thread_start_us":15955,"threads_started":1}
I20260619 15:01:22.450114 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.421163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.11.175.195:33931)
0619 15:01:22.432753 (+ 11590us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.432765 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.436168 (+ 3403us) server_negotiation.cc:407] Connection header received
0619 15:01:22.436761 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.436773 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.437120 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.437363 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:22.440729 (+ 3366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.443023 (+ 2294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.447097 (+ 4074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.448509 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.448717 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.448972 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.449554 (+ 582us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.449714 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11365,"thread_start_us":11586,"threads_started":1}
I20260619 15:01:22.450901 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.420388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:33931 (local address 127.0.0.1:44704)
0619 15:01:22.435994 (+ 15606us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:22.436020 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:22.436511 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:22.439763 (+ 3252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:22.439790 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:22.439837 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:22.440445 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.440460 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.443315 (+ 2855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.443324 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.446765 (+ 3441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.446793 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.448702 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.448708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.449071 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.449314 (+ 243us) client_negotiation.cc:769] Sending connection context
0619 15:01:22.450394 (+ 1080us) client_negotiation.cc:240] Negotiation successful
0619 15:01:22.450512 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15423,"mutex_wait_us":101,"thread_start_us":12745,"threads_started":1}
I20260619 15:01:22.455694 30068 tablet_service.cc:1467] Processing CreateTablet for tablet bc200f98ae3d492a9cf550e0e47cb297 (DEFAULT_TABLE table=test-table [id=bbe30a1c42a44123928a8e56402fe0fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:22.455694 29807 tablet_service.cc:1467] Processing CreateTablet for tablet bc200f98ae3d492a9cf550e0e47cb297 (DEFAULT_TABLE table=test-table [id=bbe30a1c42a44123928a8e56402fe0fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:22.457434 29807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc200f98ae3d492a9cf550e0e47cb297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:22.457453 30068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc200f98ae3d492a9cf550e0e47cb297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:22.457646 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.434311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:43149 (local address 127.0.0.1:54034)
0619 15:01:22.447498 (+ 13187us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:22.447514 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:22.447684 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:22.448331 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:22.448336 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:22.448366 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:22.449328 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.449344 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.451335 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.451340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.453750 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.453771 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.456870 (+ 3099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.456876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.457008 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.457034 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 15:01:22.457160 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 15:01:22.457311 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13057,"thread_start_us":12908,"threads_started":1}
I20260619 15:01:22.457749 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.434690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.11.175.194:43149)
0619 15:01:22.446340 (+ 11650us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.446347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.447646 (+ 1299us) server_negotiation.cc:407] Connection header received
0619 15:01:22.447884 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.447892 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.448122 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.448352 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:22.449572 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.451151 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.454181 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.456317 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.456510 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.456727 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.457233 (+ 506us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.457406 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11467,"thread_start_us":11313,"threads_started":1}
I20260619 15:01:22.461376 29938 tablet_service.cc:1467] Processing CreateTablet for tablet bc200f98ae3d492a9cf550e0e47cb297 (DEFAULT_TABLE table=test-table [id=bbe30a1c42a44123928a8e56402fe0fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:22.462766 29938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc200f98ae3d492a9cf550e0e47cb297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:22.473500 30152 tablet_bootstrap.cc:492] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: Bootstrap starting.
I20260619 15:01:22.474373 30151 tablet_bootstrap.cc:492] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: Bootstrap starting.
I20260619 15:01:22.480432 30151 tablet_bootstrap.cc:654] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:22.481500 30153 tablet_bootstrap.cc:492] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: Bootstrap starting.
I20260619 15:01:22.481832 30152 tablet_bootstrap.cc:654] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:22.482364 30151 log.cc:824] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:22.483278 30152 log.cc:824] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:22.485522 30151 tablet_bootstrap.cc:492] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: No bootstrap required, opened a new log
I20260619 15:01:22.485821 30151 ts_tablet_manager.cc:1397] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260619 15:01:22.485934 30153 tablet_bootstrap.cc:654] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:22.487569 30153 log.cc:824] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:22.491056 30152 tablet_bootstrap.cc:492] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: No bootstrap required, opened a new log
I20260619 15:01:22.491070 30153 tablet_bootstrap.cc:492] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: No bootstrap required, opened a new log
I20260619 15:01:22.491359 30152 ts_tablet_manager.cc:1397] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.005s
I20260619 15:01:22.491370 30153 ts_tablet_manager.cc:1397] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260619 15:01:22.496402 30151 raft_consensus.cc:348] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } }
I20260619 15:01:22.497000 30151 raft_consensus.cc:374] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:22.497184 30151 raft_consensus.cc:729] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ec040e6e0714d67ac8951d8d634d836, State: Initialized, Role: FOLLOWER
I20260619 15:01:22.497928 30151 consensus_queue.cc:260] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } }
I20260619 15:01:22.499272 30133 heartbeater.cc:503] Master 127.11.175.254:33781 was elected leader, sending a full tablet report...
I20260619 15:01:22.498965 30152 raft_consensus.cc:348] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } }
I20260619 15:01:22.499469 30152 raft_consensus.cc:374] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:22.499567 30152 raft_consensus.cc:729] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bff859ac2494d1887044d19d26d8077, State: Initialized, Role: FOLLOWER
I20260619 15:01:22.500080 30151 ts_tablet_manager.cc:1428] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260619 15:01:22.500226 30152 consensus_queue.cc:260] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } }
I20260619 15:01:22.502228 30152 ts_tablet_manager.cc:1428] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260619 15:01:22.502410 30153 raft_consensus.cc:348] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } }
I20260619 15:01:22.502980 30153 raft_consensus.cc:374] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:22.503118 30153 raft_consensus.cc:729] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284ef330b6c249d6a3945fa870dc0c8a, State: Initialized, Role: FOLLOWER
I20260619 15:01:22.503839 30153 consensus_queue.cc:260] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } }
I20260619 15:01:22.507283 30003 heartbeater.cc:503] Master 127.11.175.254:33781 was elected leader, sending a full tablet report...
I20260619 15:01:22.507491 30153 ts_tablet_manager.cc:1428] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: Time spent starting tablet: real 0.016s user 0.008s sys 0.008s
W20260619 15:01:22.610618 30134 tablet.cc:2369] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 15:01:22.615749 30004 tablet.cc:2369] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 15:01:22.629010 29873 tablet.cc:2369] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:01:22.822011 30157 raft_consensus.cc:482] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 15:01:22.822383 30157 raft_consensus.cc:504] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } }
I20260619 15:01:22.824194 30157 leader_election.cc:288] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bff859ac2494d1887044d19d26d8077 (127.11.175.193:39279), 284ef330b6c249d6a3945fa870dc0c8a (127.11.175.194:43149)
I20260619 15:01:22.829582 30158 raft_consensus.cc:482] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 15:01:22.830018 30158 raft_consensus.cc:504] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } }
I20260619 15:01:22.830487 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.824497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:43149 (local address 127.11.175.195:50415)
0619 15:01:22.824708 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:22.824739 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:22.824883 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:22.825378 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:22.825383 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:22.825406 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:22.825589 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.825597 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.826852 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.826860 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.828461 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.828474 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.829626 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.829632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.829837 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.829878 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 15:01:22.829973 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 15:01:22.830133 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":1}
I20260619 15:01:22.831162 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.824862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:50415 (local address 127.11.175.194:43149)
0619 15:01:22.825071 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.825076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.825091 (+ 15us) server_negotiation.cc:407] Connection header received
0619 15:01:22.825149 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.825153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.825217 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.825335 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:22.825746 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.826720 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.828623 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.829456 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.829637 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.829721 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.830737 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.830898 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 15:01:22.831800 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.824637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:55067 (local address 127.11.175.193:39279)
0619 15:01:22.824855 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.824860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.825449 (+ 589us) server_negotiation.cc:407] Connection header received
0619 15:01:22.825715 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.825719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.825823 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.825930 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:22.826411 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.827288 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.829108 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.830516 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.830701 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.830772 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.831423 (+ 651us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.831561 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 15:01:22.832253 30158 leader_election.cc:288] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ec040e6e0714d67ac8951d8d634d836 (127.11.175.195:33931), 284ef330b6c249d6a3945fa870dc0c8a (127.11.175.194:43149)
I20260619 15:01:22.832178 29958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc200f98ae3d492a9cf550e0e47cb297" candidate_uuid: "8ec040e6e0714d67ac8951d8d634d836" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284ef330b6c249d6a3945fa870dc0c8a" is_pre_election: true
I20260619 15:01:22.832540 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.824502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39279 (local address 127.11.175.195:55067)
0619 15:01:22.825333 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:22.825347 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:22.825553 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:22.825943 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:22.825947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:22.825969 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:22.826251 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.826260 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.827434 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.827440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.828924 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.828936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.831004 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.831010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.831205 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.831243 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 15:01:22.831992 (+ 749us) client_negotiation.cc:240] Negotiation successful
0619 15:01:22.832135 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":227,"spinlock_wait_cycles":26368,"thread_start_us":294,"threads_started":1}
I20260619 15:01:22.832844 29958 raft_consensus.cc:2454] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ec040e6e0714d67ac8951d8d634d836 in term 0.
I20260619 15:01:22.833714 30021 leader_election.cc:302] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 284ef330b6c249d6a3945fa870dc0c8a, 8ec040e6e0714d67ac8951d8d634d836; no voters:
I20260619 15:01:22.834189 30157 raft_consensus.cc:2781] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 15:01:22.834378 30157 raft_consensus.cc:482] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 15:01:22.834126 29827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc200f98ae3d492a9cf550e0e47cb297" candidate_uuid: "8ec040e6e0714d67ac8951d8d634d836" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bff859ac2494d1887044d19d26d8077" is_pre_election: true
I20260619 15:01:22.834498 30157 raft_consensus.cc:3037] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:22.834764 29827 raft_consensus.cc:2454] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ec040e6e0714d67ac8951d8d634d836 in term 0.
I20260619 15:01:22.837960 30157 raft_consensus.cc:504] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } }
I20260619 15:01:22.838799 30157 leader_election.cc:288] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [CANDIDATE]: Term 1 election: Requested vote from peers 1bff859ac2494d1887044d19d26d8077 (127.11.175.193:39279), 284ef330b6c249d6a3945fa870dc0c8a (127.11.175.194:43149)
I20260619 15:01:22.839201 29827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc200f98ae3d492a9cf550e0e47cb297" candidate_uuid: "8ec040e6e0714d67ac8951d8d634d836" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bff859ac2494d1887044d19d26d8077"
I20260619 15:01:22.839430 29827 raft_consensus.cc:3037] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:22.839474 29958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc200f98ae3d492a9cf550e0e47cb297" candidate_uuid: "8ec040e6e0714d67ac8951d8d634d836" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284ef330b6c249d6a3945fa870dc0c8a"
I20260619 15:01:22.839692 29958 raft_consensus.cc:3037] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:22.841409 30161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.832395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:33931 (local address 127.11.175.193:48455)
0619 15:01:22.833021 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:22.833041 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:22.833242 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:22.833757 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:22.833762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:22.833797 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:22.834182 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.834204 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.835822 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.835828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.838190 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.838206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.840056 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.840062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.840257 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.840307 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 15:01:22.840922 (+ 615us) client_negotiation.cc:240] Negotiation successful
0619 15:01:22.841078 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":428,"threads_started":1}
I20260619 15:01:22.840857 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.832772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:48455 (local address 127.11.175.195:33931)
0619 15:01:22.832927 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.832932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.833121 (+ 189us) server_negotiation.cc:407] Connection header received
0619 15:01:22.833415 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.833422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.833494 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.833555 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:22.834397 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.835607 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.838406 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.839657 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.839821 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.839917 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.840474 (+ 557us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.840614 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 15:01:22.842988 30088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc200f98ae3d492a9cf550e0e47cb297" candidate_uuid: "1bff859ac2494d1887044d19d26d8077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ec040e6e0714d67ac8951d8d634d836" is_pre_election: true
I20260619 15:01:22.843353 29958 raft_consensus.cc:2454] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ec040e6e0714d67ac8951d8d634d836 in term 1.
I20260619 15:01:22.843513 30088 raft_consensus.cc:2379] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bff859ac2494d1887044d19d26d8077 in current term 1: Already voted for candidate 8ec040e6e0714d67ac8951d8d634d836 in this term.
I20260619 15:01:22.843602 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.833258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:43149 (local address 127.11.175.193:43027)
0619 15:01:22.836480 (+ 3222us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:22.836496 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:22.836652 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:22.837036 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:22.837040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:22.837064 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:22.837420 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.837429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.838615 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.838621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.840928 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:22.840945 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.842751 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.842757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:22.842937 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.842981 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 15:01:22.843089 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 15:01:22.843244 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3083,"thread_start_us":2851,"threads_started":1}
I20260619 15:01:22.843886 30021 leader_election.cc:302] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 284ef330b6c249d6a3945fa870dc0c8a, 8ec040e6e0714d67ac8951d8d634d836; no voters:
I20260619 15:01:22.843918 29827 raft_consensus.cc:2454] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ec040e6e0714d67ac8951d8d634d836 in term 1.
I20260619 15:01:22.844215 30157 raft_consensus.cc:2781] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:22.844559 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.833676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:43027 (local address 127.11.175.194:43149)
0619 15:01:22.833853 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.833861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.836590 (+ 2729us) server_negotiation.cc:407] Connection header received
0619 15:01:22.836815 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.836820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.836912 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.836991 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:22.837578 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.838459 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.841171 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.842531 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.842706 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.842782 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.843136 (+ 354us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.843289 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 15:01:22.844836 30157 raft_consensus.cc:686] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 1 LEADER]: Becoming Leader. State: Replica: 8ec040e6e0714d67ac8951d8d634d836, State: Running, Role: LEADER
I20260619 15:01:22.845208 29958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc200f98ae3d492a9cf550e0e47cb297" candidate_uuid: "1bff859ac2494d1887044d19d26d8077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284ef330b6c249d6a3945fa870dc0c8a" is_pre_election: true
I20260619 15:01:22.845530 29958 raft_consensus.cc:2379] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bff859ac2494d1887044d19d26d8077 in current term 1: Already voted for candidate 8ec040e6e0714d67ac8951d8d634d836 in this term.
I20260619 15:01:22.845441 30157 consensus_queue.cc:237] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } }
I20260619 15:01:22.846256 29760 leader_election.cc:302] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bff859ac2494d1887044d19d26d8077; no voters: 284ef330b6c249d6a3945fa870dc0c8a, 8ec040e6e0714d67ac8951d8d634d836
I20260619 15:01:22.846786 30158 raft_consensus.cc:2726] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 15:01:22.851116 29684 catalog_manager.cc:5495] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ec040e6e0714d67ac8951d8d634d836 (127.11.175.195). New cstate: current_term: 1 leader_uuid: "8ec040e6e0714d67ac8951d8d634d836" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ec040e6e0714d67ac8951d8d634d836" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 } health_report { overall_health: UNKNOWN } } }
I20260619 15:01:22.896961 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.890503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.11.175.195:33931)
0619 15:01:22.890772 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.890779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.890926 (+ 147us) server_negotiation.cc:407] Connection header received
0619 15:01:22.891258 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.891262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.891387 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.891562 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:01:22.891973 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.892782 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.894405 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.894928 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.895076 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.895678 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:01:22.896241 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:01:22.896414 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.896546 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.896636 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 15:01:22.906813 29958 raft_consensus.cc:1261] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [term 1 FOLLOWER]: Refusing update from remote peer 8ec040e6e0714d67ac8951d8d634d836: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:22.906849 29827 raft_consensus.cc:1261] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 1 FOLLOWER]: Refusing update from remote peer 8ec040e6e0714d67ac8951d8d634d836: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:22.907998 30157 consensus_queue.cc:1035] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bff859ac2494d1887044d19d26d8077" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:22.908391 30163 consensus_queue.cc:1035] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [LEADER]: Connected to new peer: Peer: permanent_uuid: "284ef330b6c249d6a3945fa870dc0c8a" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 43149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:22.938824 29684 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:22.939131 29684 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:22.942968 29684 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fc748f694644ff9976df0c5af60d412: Sending DeleteTablet for 3 replicas of tablet bc200f98ae3d492a9cf550e0e47cb297
I20260619 15:01:22.944279 29807 tablet_service.cc:1514] Processing DeleteTablet for tablet bc200f98ae3d492a9cf550e0e47cb297 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:22 UTC) from {username='slave'} at 127.0.0.1:59450
I20260619 15:01:22.944468 29938 tablet_service.cc:1514] Processing DeleteTablet for tablet bc200f98ae3d492a9cf550e0e47cb297 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:22 UTC) from {username='slave'} at 127.0.0.1:54034
I20260619 15:01:22.944509 30068 tablet_service.cc:1514] Processing DeleteTablet for tablet bc200f98ae3d492a9cf550e0e47cb297 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:22 UTC) from {username='slave'} at 127.0.0.1:44704
I20260619 15:01:22.946575 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 29745
I20260619 15:01:22.955534 30177 tablet_replica.cc:330] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: stopping tablet replica
I20260619 15:01:22.955718 30176 tablet_replica.cc:330] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: stopping tablet replica
I20260619 15:01:22.956278 30176 raft_consensus.cc:2229] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:22.956287 30177 raft_consensus.cc:2229] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:22.956874 30176 raft_consensus.cc:2258] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:22.956874 30177 raft_consensus.cc:2258] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:22.957732 30178 tablet_replica.cc:330] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: stopping tablet replica
I20260619 15:01:22.958285 30178 raft_consensus.cc:2229] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 1 LEADER]: Raft consensus shutting down.
I20260619 15:01:22.960130 30176 ts_tablet_manager.cc:1905] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:22.960417 30177 ts_tablet_manager.cc:1905] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:22.961215 30178 raft_consensus.cc:2258] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:22.966364 30178 ts_tablet_manager.cc:1905] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:22.968786 30176 ts_tablet_manager.cc:1918] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:22.969102 30176 log.cc:1196] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/bc200f98ae3d492a9cf550e0e47cb297
I20260619 15:01:22.969087 30177 ts_tablet_manager.cc:1918] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:22.969290 30177 log.cc:1196] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/bc200f98ae3d492a9cf550e0e47cb297
I20260619 15:01:22.969936 30176 ts_tablet_manager.cc:1939] T bc200f98ae3d492a9cf550e0e47cb297 P 1bff859ac2494d1887044d19d26d8077: Deleting consensus metadata
I20260619 15:01:22.969969 30177 ts_tablet_manager.cc:1939] T bc200f98ae3d492a9cf550e0e47cb297 P 284ef330b6c249d6a3945fa870dc0c8a: Deleting consensus metadata
I20260619 15:01:22.971793 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:22.948442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59456 (local address 127.11.175.193:39279)
0619 15:01:22.948717 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:22.948725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:22.958072 (+ 9347us) server_negotiation.cc:407] Connection header received
0619 15:01:22.958298 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:22.958307 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:22.958447 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:22.958542 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:22.959359 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.960492 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.962520 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:22.963232 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:22.963334 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:22.969668 (+ 6334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:22.969685 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:22.969707 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:22.970624 (+ 917us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:22.970729 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:22.970775 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:22.970886 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:22.971251 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 15:01:22.971387 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260619 15:01:22.972102 29670 catalog_manager.cc:4841] TS 284ef330b6c249d6a3945fa870dc0c8a (127.11.175.194:43149): tablet bc200f98ae3d492a9cf550e0e47cb297 (table test-table [id=bbe30a1c42a44123928a8e56402fe0fe]) successfully deleted
I20260619 15:01:22.972411 29673 catalog_manager.cc:4841] TS 1bff859ac2494d1887044d19d26d8077 (127.11.175.193:39279): tablet bc200f98ae3d492a9cf550e0e47cb297 (table test-table [id=bbe30a1c42a44123928a8e56402fe0fe]) successfully deleted
I20260619 15:01:22.973341 29867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:22.978790 30178 ts_tablet_manager.cc:1918] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:22.979007 30178 log.cc:1196] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/bc200f98ae3d492a9cf550e0e47cb297
I20260619 15:01:22.979629 30178 ts_tablet_manager.cc:1939] T bc200f98ae3d492a9cf550e0e47cb297 P 8ec040e6e0714d67ac8951d8d634d836: Deleting consensus metadata
I20260619 15:01:22.981775 29670 catalog_manager.cc:4841] TS 8ec040e6e0714d67ac8951d8d634d836 (127.11.175.195:33931): tablet bc200f98ae3d492a9cf550e0e47cb297 (table test-table [id=bbe30a1c42a44123928a8e56402fe0fe]) successfully deleted
I20260619 15:01:23.068315 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 29745
I20260619 15:01:23.088333 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 29876
I20260619 15:01:23.098093 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:23.089400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54038 (local address 127.11.175.194:43149)
0619 15:01:23.089605 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:23.089610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:23.089625 (+ 15us) server_negotiation.cc:407] Connection header received
0619 15:01:23.089679 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:23.089683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:23.089778 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:23.089901 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:23.090383 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:23.091176 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:23.092853 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:23.093639 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:23.093872 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:23.095973 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:23.095991 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:23.096021 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:23.097107 (+ 1086us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:23.097167 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:23.097192 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:23.097401 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:23.097548 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 15:01:23.097669 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 15:01:23.098958 29998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:23.193820 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 29876
I20260619 15:01:23.217825 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 30006
I20260619 15:01:23.225854 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:23.218798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.11.175.195:33931)
0619 15:01:23.219005 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:23.219012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:23.219033 (+ 21us) server_negotiation.cc:407] Connection header received
0619 15:01:23.219241 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:23.219247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:23.219376 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:23.219521 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:23.219953 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:23.220813 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:23.222333 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:23.222916 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:23.223079 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:23.224348 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:23.224357 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:23.224376 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:23.224982 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:23.225053 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:23.225084 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:23.225215 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:23.225391 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 15:01:23.225499 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 15:01:23.226725 30128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:23.351593 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 30006
I20260619 15:01:23.371726 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 29655
I20260619 15:01:23.373251 29716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:23.487190 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 29655
2026-06-19T15:01:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3222 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:23 dist-test-slave-76rk krb5kdc[30186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:23 dist-test-slave-76rk krb5kdc[30186](info): set up 2 sockets
Jun 19 15:01:23 dist-test-slave-76rk krb5kdc[30186](info): commencing operation
krb5kdc: starting...
W20260619 15:01:24.619407 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:24 dist-test-slave-76rk krb5kdc[30186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:01:24.901935 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38047
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:25.264796 30199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:25.265175 30199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:25.265336 30199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:25.273085 30199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:25.273274 30199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:25.273389 30199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:25.273490 30199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:25.273576 30199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:25.284194 30199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:25.285708 30199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:25.287210 30199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:25.300283 30205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:25.300233 30204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:25.300457 30207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:25.301455 30199 server_base.cc:1034] running on GCE node
I20260619 15:01:25.302335 30199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:25.304003 30199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:25.305307 30199 hybrid_clock.cc:648] HybridClock initialized: now 1781881285305222 us; error 67 us; skew 500 ppm
I20260619 15:01:25.305719 30199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:25 dist-test-slave-76rk krb5kdc[30186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881285, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:25.309228 30199 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:01:25.310999 30199 webserver.cc:460] Webserver started at http://127.11.175.254:36977/ using document root <none> and password file <none>
I20260619 15:01:25.311684 30199 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:25.311791 30199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:25.312049 30199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:25.315267 30199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "7903c0d0b5bf4299b8c59298392c5cf6"
format_stamp: "Formatted at 2026-06-19 15:01:25 on dist-test-slave-76rk"
server_key: "b6ee70eb9602ed853c045e06b8def8e7"
server_key_iv: "80403a3a00b7983f08ab56bac949db55"
server_key_version: "encryptionkey@0"
I20260619 15:01:25.316026 30199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "7903c0d0b5bf4299b8c59298392c5cf6"
format_stamp: "Formatted at 2026-06-19 15:01:25 on dist-test-slave-76rk"
server_key: "b6ee70eb9602ed853c045e06b8def8e7"
server_key_iv: "80403a3a00b7983f08ab56bac949db55"
server_key_version: "encryptionkey@0"
I20260619 15:01:25.321841 30199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 15:01:25.325917 30214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:25.327730 30199 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:01:25.327890 30199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "7903c0d0b5bf4299b8c59298392c5cf6"
format_stamp: "Formatted at 2026-06-19 15:01:25 on dist-test-slave-76rk"
server_key: "b6ee70eb9602ed853c045e06b8def8e7"
server_key_iv: "80403a3a00b7983f08ab56bac949db55"
server_key_version: "encryptionkey@0"
I20260619 15:01:25.328076 30199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:25.346128 30199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:25.350948 30199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:25.351214 30199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:25.372419 30199 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:38047
I20260619 15:01:25.372421 30265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:38047 every 8 connection(s)
I20260619 15:01:25.374133 30199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:25.378386 30266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:25.380658 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 30199
I20260619 15:01:25.380885 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:25.381470 11967 external_mini_cluster.cc:1428] Setting key 9cc45ac1bc28c7af162e742c92f4d2cd
I20260619 15:01:25.393589 30266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6: Bootstrap starting.
Jun 19 15:01:25 dist-test-slave-76rk krb5kdc[30186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:25.397549 30266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:25.398806 30266 log.cc:824] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:25.402017 30266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6: No bootstrap required, opened a new log
I20260619 15:01:25.407362 30269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:25.383787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57936 (local address 127.11.175.254:38047)
0619 15:01:25.384665 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:25.384677 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:25.384732 (+ 55us) server_negotiation.cc:407] Connection header received
0619 15:01:25.386510 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:25.386538 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:25.387298 (+ 760us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:25.387745 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:25.389432 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:25.390988 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:25.392494 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:25.393273 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:25.393492 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:25.398569 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:25.398578 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:25.398598 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:25.398715 (+ 117us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:25.402052 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:25.403043 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:25.403059 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:25.403067 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:25.403273 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:25.403769 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:25.403776 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:25.403778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:25.404386 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:25.404741 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:25.405287 (+ 546us) server_negotiation.cc:299] Negotiation successful
0619 15:01:25.405810 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":399,"threads_started":1}
I20260619 15:01:25.411897 30266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7903c0d0b5bf4299b8c59298392c5cf6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38047 } }
I20260619 15:01:25.412660 30266 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:25.412799 30266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7903c0d0b5bf4299b8c59298392c5cf6, State: Initialized, Role: FOLLOWER
I20260619 15:01:25.413591 30266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7903c0d0b5bf4299b8c59298392c5cf6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38047 } }
I20260619 15:01:25.413906 30266 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:25.414074 30266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:25.414311 30266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:25.416817 30266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7903c0d0b5bf4299b8c59298392c5cf6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38047 } }
I20260619 15:01:25.417447 30266 leader_election.cc:302] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7903c0d0b5bf4299b8c59298392c5cf6; no voters:
I20260619 15:01:25.418200 30266 leader_election.cc:288] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:25.418424 30271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:25.419194 30271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [term 1 LEADER]: Becoming Leader. State: Replica: 7903c0d0b5bf4299b8c59298392c5cf6, State: Running, Role: LEADER
I20260619 15:01:25.419934 30271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7903c0d0b5bf4299b8c59298392c5cf6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38047 } }
I20260619 15:01:25.420989 30266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:25.423207 30272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7903c0d0b5bf4299b8c59298392c5cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7903c0d0b5bf4299b8c59298392c5cf6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38047 } } }
I20260619 15:01:25.423236 30273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7903c0d0b5bf4299b8c59298392c5cf6. Latest consensus state: current_term: 1 leader_uuid: "7903c0d0b5bf4299b8c59298392c5cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7903c0d0b5bf4299b8c59298392c5cf6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38047 } } }
I20260619 15:01:25.423823 30272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:25.423831 30273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:25.426142 30277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:25.431449 30277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:25.443114 30277 catalog_manager.cc:1347] Generated new cluster ID: 95f259e9cc034df1a42631a52f58089f
I20260619 15:01:25.443250 30277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:25.485314 30277 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:25.486853 30277 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:25.501103 30277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6: Generated new TSK 0
I20260619 15:01:25.502044 30277 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 15:01:25.567145 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38047
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:25.928807 30292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:25.929104 30292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:25.929250 30292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:25.937217 30292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:25.937364 30292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:25.937558 30292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 15:01:25.948292 30292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:25.949594 30292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:25.951066 30292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:25.965377 30297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:25.965389 30298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:25.965494 30300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:25.966696 30292 server_base.cc:1034] running on GCE node
I20260619 15:01:25.967491 30292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:25.968595 30292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:25.969918 30292 hybrid_clock.cc:648] HybridClock initialized: now 1781881285969872 us; error 80 us; skew 500 ppm
I20260619 15:01:25.970331 30292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:25 dist-test-slave-76rk krb5kdc[30186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881285, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:25.974016 30292 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 15:01:25.975759 30292 webserver.cc:460] Webserver started at http://127.11.175.193:43843/ using document root <none> and password file <none>
I20260619 15:01:25.976540 30292 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:25.976703 30292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:25.977056 30292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:25.980273 30292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "9c83e86328f841f78d42d38d9c8114ab"
format_stamp: "Formatted at 2026-06-19 15:01:25 on dist-test-slave-76rk"
server_key: "b55c7fc792cfeb171baaaf6166634163"
server_key_iv: "32845a23c7db6f9854762ce6299d306a"
server_key_version: "encryptionkey@0"
I20260619 15:01:25.981153 30292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "9c83e86328f841f78d42d38d9c8114ab"
format_stamp: "Formatted at 2026-06-19 15:01:25 on dist-test-slave-76rk"
server_key: "b55c7fc792cfeb171baaaf6166634163"
server_key_iv: "32845a23c7db6f9854762ce6299d306a"
server_key_version: "encryptionkey@0"
I20260619 15:01:25.987156 30292 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260619 15:01:25.991334 30307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:25.993031 30292 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 15:01:25.993240 30292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "9c83e86328f841f78d42d38d9c8114ab"
format_stamp: "Formatted at 2026-06-19 15:01:25 on dist-test-slave-76rk"
server_key: "b55c7fc792cfeb171baaaf6166634163"
server_key_iv: "32845a23c7db6f9854762ce6299d306a"
server_key_version: "encryptionkey@0"
I20260619 15:01:25.993515 30292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:26.010588 30292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:26.016000 30292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:26.016338 30292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:26.017478 30292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:26.019294 30292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:26.019390 30292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:26.019536 30292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:26.019601 30292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:26.059089 30292 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:46209
I20260619 15:01:26.059110 30419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:46209 every 8 connection(s)
I20260619 15:01:26.060694 30292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 15:01:26.065480 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 30292
I20260619 15:01:26.065697 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 15:01:26.066203 11967 external_mini_cluster.cc:1428] Setting key 9f7655edb8e5c13d3180854b4c496b49
Jun 19 15:01:26 dist-test-slave-76rk krb5kdc[30186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881285, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:26.085318 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:26.063837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:45899 (local address 127.11.175.254:38047)
0619 15:01:26.064401 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:26.064406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:26.066733 (+ 2327us) server_negotiation.cc:407] Connection header received
0619 15:01:26.068042 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:26.068047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:26.068139 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:26.068270 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:26.069978 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:26.070839 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:26.072616 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:26.073044 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:26.073221 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:26.077943 (+ 4722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:26.077949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:26.077952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:26.077998 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:26.081022 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:26.082230 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:26.082235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:26.082238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:26.082417 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:26.083123 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:26.083128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:26.083130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:26.083517 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:26.083666 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:26.084785 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0619 15:01:26.084993 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":310,"threads_started":1}
I20260619 15:01:26.086953 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:26.064369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38047 (local address 127.11.175.193:45899)
0619 15:01:26.065431 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:26.065482 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:26.067735 (+ 2253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:26.068540 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:26.068550 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:26.069122 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:26.069762 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:26.069788 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:26.071036 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:26.071041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:26.072420 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:26.072434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:26.073336 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:26.073345 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:26.073621 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:26.074449 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:01:26.074475 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:26.077585 (+ 3110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:26.081240 (+ 3655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:26.081254 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:26.081289 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:26.082049 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:26.082569 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:26.082577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:26.082584 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:26.082933 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:26.083680 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:26.083692 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:26.084245 (+ 553us) client_negotiation.cc:769] Sending connection context
0619 15:01:26.084607 (+ 362us) client_negotiation.cc:240] Negotiation successful
0619 15:01:26.085283 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":398,"threads_started":1}
I20260619 15:01:26.088864 30420 heartbeater.cc:347] Connected to a master server at 127.11.175.254:38047
I20260619 15:01:26.089249 30420 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:26.090137 30420 heartbeater.cc:511] Master 127.11.175.254:38047 requested a full tablet report, sending...
I20260619 15:01:26.092523 30231 ts_manager.cc:194] Registered new tserver with Master: 9c83e86328f841f78d42d38d9c8114ab (127.11.175.193:46209)
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 15:01:26.094169 30231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:45899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 15:01:26.131248 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38047
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:26.486135 30426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:26.486531 30426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:26.486747 30426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:26.494423 30426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:26.494611 30426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:26.494822 30426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 15:01:26.506184 30426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:26.507745 30426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:26.509249 30426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:26.523571 30434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:26.523653 30432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:26.523612 30431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:26.525053 30426 server_base.cc:1034] running on GCE node
I20260619 15:01:26.525805 30426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:26.526989 30426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:26.528245 30426 hybrid_clock.cc:648] HybridClock initialized: now 1781881286528198 us; error 58 us; skew 500 ppm
I20260619 15:01:26.528664 30426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:26 dist-test-slave-76rk krb5kdc[30186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:26.532505 30426 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 15:01:26.534173 30426 webserver.cc:460] Webserver started at http://127.11.175.194:35371/ using document root <none> and password file <none>
I20260619 15:01:26.534972 30426 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:26.535120 30426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:26.535446 30426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:26.538399 30426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "7d6dda24739f4145b8c8e579e928b333"
format_stamp: "Formatted at 2026-06-19 15:01:26 on dist-test-slave-76rk"
server_key: "567a19a74b9c1a2c0883ca2e7fd7fa9f"
server_key_iv: "69ce9223de9cf5b62b060bfe4b4be7b2"
server_key_version: "encryptionkey@0"
I20260619 15:01:26.539251 30426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "7d6dda24739f4145b8c8e579e928b333"
format_stamp: "Formatted at 2026-06-19 15:01:26 on dist-test-slave-76rk"
server_key: "567a19a74b9c1a2c0883ca2e7fd7fa9f"
server_key_iv: "69ce9223de9cf5b62b060bfe4b4be7b2"
server_key_version: "encryptionkey@0"
I20260619 15:01:26.544545 30426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 15:01:26.548171 30441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:26.549547 30426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 15:01:26.549727 30426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "7d6dda24739f4145b8c8e579e928b333"
format_stamp: "Formatted at 2026-06-19 15:01:26 on dist-test-slave-76rk"
server_key: "567a19a74b9c1a2c0883ca2e7fd7fa9f"
server_key_iv: "69ce9223de9cf5b62b060bfe4b4be7b2"
server_key_version: "encryptionkey@0"
I20260619 15:01:26.549944 30426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:26.561375 30426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:26.566110 30426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:26.566447 30426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:26.567595 30426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:26.569072 30426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:26.569183 30426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:26.569339 30426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:26.569413 30426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:26.605674 30426 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:44301
I20260619 15:01:26.605762 30553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:44301 every 8 connection(s)
I20260619 15:01:26.607131 30426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 15:01:26.609369 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 30426
I20260619 15:01:26.609560 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 15:01:26.610160 11967 external_mini_cluster.cc:1428] Setting key 7c50338d61b6300622a9e00455fdd0b5
Jun 19 15:01:26 dist-test-slave-76rk krb5kdc[30186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:26.632679 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:26.609846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:38039 (local address 127.11.175.254:38047)
0619 15:01:26.610360 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:26.610367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:26.611482 (+ 1115us) server_negotiation.cc:407] Connection header received
0619 15:01:26.614124 (+ 2642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:26.614129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:26.614209 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:26.614312 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:26.616586 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:26.617406 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:26.620075 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:26.620526 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:26.620672 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:26.624874 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:26.624884 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:26.624891 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:26.624967 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:26.628904 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:26.629819 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:26.629823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:26.629826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:26.629969 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:26.630526 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:26.630530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:26.630532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:26.630938 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:26.631101 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:26.631942 (+ 841us) server_negotiation.cc:299] Negotiation successful
0619 15:01:26.632250 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":262,"threads_started":1}
I20260619 15:01:26.633823 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:26.610425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38047 (local address 127.11.175.194:38039)
0619 15:01:26.611320 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:26.611376 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:26.613777 (+ 2401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:26.614623 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:26.614666 (+ 43us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:26.615512 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:26.616369 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:26.616397 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:26.617585 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:26.617593 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:26.619892 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:26.619911 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:26.620790 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:26.620800 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:26.621177 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:26.621965 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:01:26.622000 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:26.624565 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:26.629085 (+ 4520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:26.629093 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:26.629108 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:26.629631 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:26.630102 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:26.630107 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:26.630118 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:26.630379 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:26.631080 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:26.631088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:26.631547 (+ 459us) client_negotiation.cc:769] Sending connection context
0619 15:01:26.631842 (+ 295us) client_negotiation.cc:240] Negotiation successful
0619 15:01:26.632547 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":375,"threads_started":1}
I20260619 15:01:26.635517 30554 heartbeater.cc:347] Connected to a master server at 127.11.175.254:38047
I20260619 15:01:26.635907 30554 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:26.636660 30554 heartbeater.cc:511] Master 127.11.175.254:38047 requested a full tablet report, sending...
I20260619 15:01:26.638325 30231 ts_manager.cc:194] Registered new tserver with Master: 7d6dda24739f4145b8c8e579e928b333 (127.11.175.194:44301)
I20260619 15:01:26.639678 30231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:38039
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 15:01:26.674319 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38047
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:27.024199 30560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:27.024559 30560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:27.024736 30560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:27.032449 30560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:27.032609 30560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:27.032799 30560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 15:01:27.043066 30560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:38047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:27.044431 30560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:27.045779 30560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:27.061246 30566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:27.061784 30565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:27.062065 30568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:27.066471 30560 server_base.cc:1034] running on GCE node
I20260619 15:01:27.067332 30560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:27.068382 30560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:27.069718 30560 hybrid_clock.cc:648] HybridClock initialized: now 1781881287069659 us; error 48 us; skew 500 ppm
I20260619 15:01:27.070190 30560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:27 dist-test-slave-76rk krb5kdc[30186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881287, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:27.073804 30560 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 15:01:27.075876 30560 webserver.cc:460] Webserver started at http://127.11.175.195:42217/ using document root <none> and password file <none>
I20260619 15:01:27.076886 30560 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:27.077050 30560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:27.077409 30560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:27.080690 30560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "c946147e3d7e4feebea827aef9ecbd63"
format_stamp: "Formatted at 2026-06-19 15:01:27 on dist-test-slave-76rk"
server_key: "491af8ec45c0d125895529486de45103"
server_key_iv: "9d8a9094f791a3097e73d5c0830d05b6"
server_key_version: "encryptionkey@0"
I20260619 15:01:27.081548 30560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "c946147e3d7e4feebea827aef9ecbd63"
format_stamp: "Formatted at 2026-06-19 15:01:27 on dist-test-slave-76rk"
server_key: "491af8ec45c0d125895529486de45103"
server_key_iv: "9d8a9094f791a3097e73d5c0830d05b6"
server_key_version: "encryptionkey@0"
I20260619 15:01:27.087280 30560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 15:01:27.091286 30575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:27.092756 30560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 15:01:27.092964 30560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "c946147e3d7e4feebea827aef9ecbd63"
format_stamp: "Formatted at 2026-06-19 15:01:27 on dist-test-slave-76rk"
server_key: "491af8ec45c0d125895529486de45103"
server_key_iv: "9d8a9094f791a3097e73d5c0830d05b6"
server_key_version: "encryptionkey@0"
I20260619 15:01:27.093190 30560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:27.097798 30420 heartbeater.cc:503] Master 127.11.175.254:38047 was elected leader, sending a full tablet report...
I20260619 15:01:27.127813 30560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:27.132937 30560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:27.133288 30560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:27.134361 30560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:27.135957 30560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:27.136034 30560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:27.136169 30560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:27.136229 30560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:27.169802 30560 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:40111
I20260619 15:01:27.169852 30687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:40111 every 8 connection(s)
I20260619 15:01:27.171319 30560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 15:01:27.172448 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 30560
I20260619 15:01:27.172660 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 15:01:27.173231 11967 external_mini_cluster.cc:1428] Setting key 6330d2c66feafb0fa37f036247ce7b29
Jun 19 15:01:27 dist-test-slave-76rk krb5kdc[30186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881287, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:27.194504 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.174500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:35621 (local address 127.11.175.254:38047)
0619 15:01:27.175203 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.175209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.176602 (+ 1393us) server_negotiation.cc:407] Connection header received
0619 15:01:27.178339 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.178343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.178414 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.178515 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:27.180102 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.180924 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.182540 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.183037 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.183148 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.187440 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:27.187449 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:27.187457 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:27.187521 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:27.191107 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:27.192036 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:27.192041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:27.192044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:27.192194 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:27.192747 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:27.192752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:27.192754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:27.193061 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:27.193189 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.193975 (+ 786us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.194172 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":465,"threads_started":1}
I20260619 15:01:27.195753 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.175298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:38047 (local address 127.11.175.195:35621)
0619 15:01:27.176228 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.176291 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:27.178063 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:27.178812 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:27.178822 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:27.179351 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:27.179938 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.179956 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.181085 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.181089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.182376 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.182389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.183431 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.183438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.183664 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.184432 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:01:27.184459 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:27.187142 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:27.191291 (+ 4149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:27.191300 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:27.191317 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:27.191868 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:27.192374 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:27.192378 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:27.192380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:27.192599 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:27.193184 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:27.193191 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:27.193558 (+ 367us) client_negotiation.cc:769] Sending connection context
0619 15:01:27.193824 (+ 266us) client_negotiation.cc:240] Negotiation successful
0619 15:01:27.194254 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":379,"threads_started":1}
I20260619 15:01:27.197753 30688 heartbeater.cc:347] Connected to a master server at 127.11.175.254:38047
I20260619 15:01:27.198163 30688 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:27.199085 30688 heartbeater.cc:511] Master 127.11.175.254:38047 requested a full tablet report, sending...
I20260619 15:01:27.200747 30231 ts_manager.cc:194] Registered new tserver with Master: c946147e3d7e4feebea827aef9ecbd63 (127.11.175.195:40111)
I20260619 15:01:27.201792 30231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:35621
I20260619 15:01:27.202831 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 15:01:27.222833 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.207953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57940 (local address 127.11.175.254:38047)
0619 15:01:27.208161 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.208166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.208649 (+ 483us) server_negotiation.cc:407] Connection header received
0619 15:01:27.208801 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.208805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.208892 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.208990 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:27.210500 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.211348 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.212724 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.213167 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.213284 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.216547 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:27.216552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:27.216556 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:27.216597 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:27.219342 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:27.220326 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:27.220330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:27.220332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:27.220481 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:27.221046 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:27.221050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:27.221052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:27.221543 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:27.221774 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.222299 (+ 525us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.222518 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 15:01:27.227788 30230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 15:01:27.235373 30230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 15:01:27.269088 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.248555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:44301 (local address 127.0.0.1:46030)
0619 15:01:27.259743 (+ 11188us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.259763 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:27.259914 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:27.260870 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:27.260874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:27.260903 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:27.261504 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.261522 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.263113 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.263119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.265645 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.265660 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.267862 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.267870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.268126 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.268197 (+ 71us) client_negotiation.cc:769] Sending connection context
0619 15:01:27.268358 (+ 161us) client_negotiation.cc:240] Negotiation successful
0619 15:01:27.268606 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11058,"mutex_wait_us":110,"thread_start_us":11367,"threads_started":1}
I20260619 15:01:27.269520 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.250379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.11.175.195:40111)
0619 15:01:27.259039 (+ 8660us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.259049 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.260346 (+ 1297us) server_negotiation.cc:407] Connection header received
0619 15:01:27.260658 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.260666 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.260888 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.261068 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:27.262153 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.263747 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.266497 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.267835 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.268035 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.268240 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.268838 (+ 598us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.269037 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8470,"thread_start_us":8300,"threads_started":1}
I20260619 15:01:27.269520 30705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.249547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.11.175.194:44301)
0619 15:01:27.258487 (+ 8940us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.258500 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.260068 (+ 1568us) server_negotiation.cc:407] Connection header received
0619 15:01:27.260189 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.260196 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.260429 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.260623 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:27.261736 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.262967 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.265835 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.267310 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.267507 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.267701 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.269011 (+ 1310us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.269187 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8700,"thread_start_us":8551,"threads_started":1}
I20260619 15:01:27.269909 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.248683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.11.175.193:46209)
0619 15:01:27.259390 (+ 10707us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.259401 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.259555 (+ 154us) server_negotiation.cc:407] Connection header received
0619 15:01:27.259691 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.259700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.260007 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.260220 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:27.261370 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.262966 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.266398 (+ 3432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.268031 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.268234 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.268422 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.269187 (+ 765us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.269559 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10540,"thread_start_us":10414,"threads_started":1}
I20260619 15:01:27.270038 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.248468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:46209 (local address 127.0.0.1:33496)
0619 15:01:27.257373 (+ 8905us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.257445 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:27.257723 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:27.260525 (+ 2802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:27.260536 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:27.260588 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:27.261150 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.261169 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.263132 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.263138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.266183 (+ 3045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.266205 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.268576 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.268581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.268775 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.268810 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 15:01:27.268902 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 15:01:27.269058 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8525,"mutex_wait_us":261,"thread_start_us":10976,"threads_started":1}
I20260619 15:01:27.270589 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.249080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:40111 (local address 127.0.0.1:55898)
0619 15:01:27.260208 (+ 11128us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.260227 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:27.260460 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:27.261406 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:27.261409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:27.261466 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:27.261941 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.261959 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.263924 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.263956 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.266321 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.266333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.268391 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.268397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.268605 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.268639 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 15:01:27.270059 (+ 1420us) client_negotiation.cc:240] Negotiation successful
0619 15:01:27.270215 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11010,"mutex_wait_us":5,"thread_start_us":7711,"threads_started":1}
I20260619 15:01:27.273901 30355 tablet_service.cc:1467] Processing CreateTablet for tablet 4bdd7e639f3b426bb3caaf6c76fb8311 (DEFAULT_TABLE table=test-table [id=fe31f885c358402595e91270aa57dc59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:27.274008 30623 tablet_service.cc:1467] Processing CreateTablet for tablet 4bdd7e639f3b426bb3caaf6c76fb8311 (DEFAULT_TABLE table=test-table [id=fe31f885c358402595e91270aa57dc59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:27.273926 30489 tablet_service.cc:1467] Processing CreateTablet for tablet 4bdd7e639f3b426bb3caaf6c76fb8311 (DEFAULT_TABLE table=test-table [id=fe31f885c358402595e91270aa57dc59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:27.275377 30355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bdd7e639f3b426bb3caaf6c76fb8311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:27.275439 30623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bdd7e639f3b426bb3caaf6c76fb8311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:27.275441 30489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bdd7e639f3b426bb3caaf6c76fb8311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:27.288122 30707 tablet_bootstrap.cc:492] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab: Bootstrap starting.
I20260619 15:01:27.291095 30708 tablet_bootstrap.cc:492] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: Bootstrap starting.
I20260619 15:01:27.292661 30707 tablet_bootstrap.cc:654] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:27.294354 30707 log.cc:824] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:27.295468 30709 tablet_bootstrap.cc:492] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: Bootstrap starting.
I20260619 15:01:27.295789 30708 tablet_bootstrap.cc:654] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:27.297797 30708 log.cc:824] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:27.299559 30709 tablet_bootstrap.cc:654] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:27.300669 30709 log.cc:824] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:27.301530 30708 tablet_bootstrap.cc:492] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: No bootstrap required, opened a new log
I20260619 15:01:27.301908 30708 ts_tablet_manager.cc:1397] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.003s
I20260619 15:01:27.303486 30707 tablet_bootstrap.cc:492] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab: No bootstrap required, opened a new log
I20260619 15:01:27.303789 30707 ts_tablet_manager.cc:1397] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.000s
I20260619 15:01:27.305321 30709 tablet_bootstrap.cc:492] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: No bootstrap required, opened a new log
I20260619 15:01:27.305601 30709 ts_tablet_manager.cc:1397] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260619 15:01:27.313308 30709 raft_consensus.cc:348] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c83e86328f841f78d42d38d9c8114ab" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 46209 } } peers { permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 } } peers { permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 } }
I20260619 15:01:27.313472 30708 raft_consensus.cc:348] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c83e86328f841f78d42d38d9c8114ab" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 46209 } } peers { permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 } } peers { permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 } }
I20260619 15:01:27.314019 30709 raft_consensus.cc:374] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:27.314133 30708 raft_consensus.cc:374] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:27.314196 30709 raft_consensus.cc:729] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c946147e3d7e4feebea827aef9ecbd63, State: Initialized, Role: FOLLOWER
I20260619 15:01:27.314306 30708 raft_consensus.cc:729] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6dda24739f4145b8c8e579e928b333, State: Initialized, Role: FOLLOWER
I20260619 15:01:27.314950 30709 consensus_queue.cc:260] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c83e86328f841f78d42d38d9c8114ab" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 46209 } } peers { permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 } } peers { permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 } }
I20260619 15:01:27.315133 30708 consensus_queue.cc:260] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c83e86328f841f78d42d38d9c8114ab" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 46209 } } peers { permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 } } peers { permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 } }
I20260619 15:01:27.314978 30707 raft_consensus.cc:348] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c83e86328f841f78d42d38d9c8114ab" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 46209 } } peers { permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 } } peers { permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 } }
I20260619 15:01:27.315606 30707 raft_consensus.cc:374] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:27.315779 30707 raft_consensus.cc:729] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c83e86328f841f78d42d38d9c8114ab, State: Initialized, Role: FOLLOWER
I20260619 15:01:27.316211 30688 heartbeater.cc:503] Master 127.11.175.254:38047 was elected leader, sending a full tablet report...
I20260619 15:01:27.316956 30709 ts_tablet_manager.cc:1428] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260619 15:01:27.316860 30707 consensus_queue.cc:260] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c83e86328f841f78d42d38d9c8114ab" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 46209 } } peers { permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 } } peers { permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 } }
I20260619 15:01:27.319108 30707 ts_tablet_manager.cc:1428] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260619 15:01:27.319204 30554 heartbeater.cc:503] Master 127.11.175.254:38047 was elected leader, sending a full tablet report...
I20260619 15:01:27.320397 30708 ts_tablet_manager.cc:1428] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: Time spent starting tablet: real 0.018s user 0.011s sys 0.006s
I20260619 15:01:27.320402 30715 raft_consensus.cc:482] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 15:01:27.320647 30715 raft_consensus.cc:504] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c83e86328f841f78d42d38d9c8114ab" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 46209 } } peers { permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 } } peers { permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 } }
I20260619 15:01:27.325213 30715 leader_election.cc:288] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d6dda24739f4145b8c8e579e928b333 (127.11.175.194:44301), c946147e3d7e4feebea827aef9ecbd63 (127.11.175.195:40111)
I20260619 15:01:27.330590 30705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.323573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:43127 (local address 127.11.175.194:44301)
0619 15:01:27.323782 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.323788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.324064 (+ 276us) server_negotiation.cc:407] Connection header received
0619 15:01:27.324369 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.324374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.324468 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.324535 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:27.325423 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.326440 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.328761 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.329585 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.329781 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.329864 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.330173 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.330322 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 15:01:27.330686 30716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.323118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:44301 (local address 127.11.175.193:43127)
0619 15:01:27.323958 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.323981 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:27.324203 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:27.324738 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:27.324743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:27.324774 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:27.325237 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.325258 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.326586 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.326591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.328575 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.328589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.329813 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.329819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.329996 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.330022 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 15:01:27.330130 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 15:01:27.330278 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":490,"threads_started":1}
I20260619 15:01:27.331830 30509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bdd7e639f3b426bb3caaf6c76fb8311" candidate_uuid: "9c83e86328f841f78d42d38d9c8114ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6dda24739f4145b8c8e579e928b333" is_pre_election: true
I20260619 15:01:27.332353 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.324553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:42615 (local address 127.11.175.195:40111)
0619 15:01:27.324755 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.324760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.325793 (+ 1033us) server_negotiation.cc:407] Connection header received
0619 15:01:27.326035 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.326040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.326114 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.326218 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:27.327027 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.327953 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.329752 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.330569 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.330807 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.330892 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.331702 (+ 810us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.331879 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 15:01:27.332743 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.324291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:40111 (local address 127.11.175.193:42615)
0619 15:01:27.325675 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.325714 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:27.325890 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:27.326229 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:27.326234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:27.326257 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:27.326826 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.326845 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.328101 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.328108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.329598 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:27.329612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.330872 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.330879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:27.331509 (+ 630us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.331554 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 15:01:27.331676 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 15:01:27.331912 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1270,"thread_start_us":1087,"threads_started":1}
I20260619 15:01:27.332898 30509 raft_consensus.cc:2454] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c83e86328f841f78d42d38d9c8114ab in term 0.
I20260619 15:01:27.333715 30310 leader_election.cc:302] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6dda24739f4145b8c8e579e928b333, 9c83e86328f841f78d42d38d9c8114ab; no voters:
I20260619 15:01:27.333861 30643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bdd7e639f3b426bb3caaf6c76fb8311" candidate_uuid: "9c83e86328f841f78d42d38d9c8114ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c946147e3d7e4feebea827aef9ecbd63" is_pre_election: true
I20260619 15:01:27.334213 30715 raft_consensus.cc:2781] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 15:01:27.334384 30715 raft_consensus.cc:482] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 15:01:27.334506 30715 raft_consensus.cc:3037] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:27.334463 30643 raft_consensus.cc:2454] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c83e86328f841f78d42d38d9c8114ab in term 0.
I20260619 15:01:27.336952 30715 raft_consensus.cc:504] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c83e86328f841f78d42d38d9c8114ab" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 46209 } } peers { permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 } } peers { permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 } }
I20260619 15:01:27.337587 30715 leader_election.cc:288] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [CANDIDATE]: Term 1 election: Requested vote from peers 7d6dda24739f4145b8c8e579e928b333 (127.11.175.194:44301), c946147e3d7e4feebea827aef9ecbd63 (127.11.175.195:40111)
I20260619 15:01:27.338128 30509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bdd7e639f3b426bb3caaf6c76fb8311" candidate_uuid: "9c83e86328f841f78d42d38d9c8114ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6dda24739f4145b8c8e579e928b333"
I20260619 15:01:27.338151 30643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bdd7e639f3b426bb3caaf6c76fb8311" candidate_uuid: "9c83e86328f841f78d42d38d9c8114ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c946147e3d7e4feebea827aef9ecbd63"
I20260619 15:01:27.338398 30509 raft_consensus.cc:3037] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:27.338428 30643 raft_consensus.cc:3037] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:27.340732 30509 raft_consensus.cc:2454] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c83e86328f841f78d42d38d9c8114ab in term 1.
I20260619 15:01:27.340775 30643 raft_consensus.cc:2454] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c83e86328f841f78d42d38d9c8114ab in term 1.
I20260619 15:01:27.341203 30310 leader_election.cc:302] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6dda24739f4145b8c8e579e928b333, 9c83e86328f841f78d42d38d9c8114ab; no voters:
I20260619 15:01:27.341548 30715 raft_consensus.cc:2781] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:27.341981 30715 raft_consensus.cc:686] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 1 LEADER]: Becoming Leader. State: Replica: 9c83e86328f841f78d42d38d9c8114ab, State: Running, Role: LEADER
I20260619 15:01:27.342578 30715 consensus_queue.cc:237] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c83e86328f841f78d42d38d9c8114ab" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 46209 } } peers { permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 } } peers { permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 } }
I20260619 15:01:27.347698 30230 catalog_manager.cc:5495] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c83e86328f841f78d42d38d9c8114ab (127.11.175.193). New cstate: current_term: 1 leader_uuid: "9c83e86328f841f78d42d38d9c8114ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c83e86328f841f78d42d38d9c8114ab" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 46209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 } health_report { overall_health: UNKNOWN } } }
W20260619 15:01:27.359226 30555 tablet.cc:2369] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:01:27.377242 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.371101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33508 (local address 127.11.175.193:46209)
0619 15:01:27.371383 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.371390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.371413 (+ 23us) server_negotiation.cc:407] Connection header received
0619 15:01:27.371595 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.371599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.371688 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.371852 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:01:27.372271 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.372997 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.374708 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.375158 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.375314 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.375992 (+ 678us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:01:27.376549 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:01:27.376760 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.376941 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.376993 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260619 15:01:27.387842 30643 raft_consensus.cc:1261] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63 [term 1 FOLLOWER]: Refusing update from remote peer 9c83e86328f841f78d42d38d9c8114ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:27.387883 30509 raft_consensus.cc:1261] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333 [term 1 FOLLOWER]: Refusing update from remote peer 9c83e86328f841f78d42d38d9c8114ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:27.389035 30718 consensus_queue.cc:1035] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d6dda24739f4145b8c8e579e928b333" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 44301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:27.389495 30715 consensus_queue.cc:1035] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c946147e3d7e4feebea827aef9ecbd63" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 40111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:27.419679 30230 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:27.419977 30230 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:27.424060 30230 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7903c0d0b5bf4299b8c59298392c5cf6: Sending DeleteTablet for 3 replicas of tablet 4bdd7e639f3b426bb3caaf6c76fb8311
I20260619 15:01:27.425326 30623 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bdd7e639f3b426bb3caaf6c76fb8311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:27 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:55898
I20260619 15:01:27.425333 30355 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bdd7e639f3b426bb3caaf6c76fb8311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:27 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:33496
I20260619 15:01:27.426127 30489 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bdd7e639f3b426bb3caaf6c76fb8311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:27 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:46030
I20260619 15:01:27.427251 30733 tablet_replica.cc:330] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: stopping tablet replica
I20260619 15:01:27.427779 30733 raft_consensus.cc:2229] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:27.428364 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 30292
I20260619 15:01:27.428443 30733 raft_consensus.cc:2258] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:27.440472 30733 ts_tablet_manager.cc:1905] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 15:01:27 dist-test-slave-76rk krb5kdc[30186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 15:01:27.449040 30731 tablet_replica.cc:330] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: stopping tablet replica
I20260619 15:01:27.449553 30731 raft_consensus.cc:2229] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:27.450130 30731 raft_consensus.cc:2258] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:27.453331 30733 ts_tablet_manager.cc:1918] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:27.453599 30733 log.cc:1196] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/4bdd7e639f3b426bb3caaf6c76fb8311
I20260619 15:01:27.453936 30731 ts_tablet_manager.cc:1905] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:27.454511 30732 tablet_replica.cc:330] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab: stopping tablet replica
I20260619 15:01:27.454881 30733 ts_tablet_manager.cc:1939] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 7d6dda24739f4145b8c8e579e928b333: Deleting consensus metadata
I20260619 15:01:27.455025 30732 raft_consensus.cc:2229] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 1 LEADER]: Raft consensus shutting down.
I20260619 15:01:27.455731 30732 raft_consensus.cc:2258] T 4bdd7e639f3b426bb3caaf6c76fb8311 P 9c83e86328f841f78d42d38d9c8114ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:27.456532 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.430402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33516 (local address 127.11.175.193:46209)
0619 15:01:27.430675 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.430685 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.430712 (+ 27us) server_negotiation.cc:407] Connection header received
0619 15:01:27.430793 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.430802 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.430903 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.431020 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:27.433749 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.435438 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.438275 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.439318 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.439455 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.446929 (+ 7474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:27.446951 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:27.446978 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:27.447044 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:27.452344 (+ 5300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:27.453378 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:27.453389 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:27.453392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:27.453621 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:27.454303 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:27.454308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:27.454311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:27.454911 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:27.455224 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.455867 (+ 643us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.456130 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":89}
I20260619 15:01:27.458089 30217 catalog_manager.cc:4841] TS 7d6dda24739f4145b8c8e579e928b333 (127.11.175.194:44301): tablet 4bdd7e639f3b426bb3caaf6c76fb8311 (table test-table [id=fe31f885c358402595e91270aa57dc59]) successfully deleted
I20260619 15:01:27.458202 30415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:27.462430 30731 ts_tablet_manager.cc:1918] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:27.462643 30731 log.cc:1196] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/4bdd7e639f3b426bb3caaf6c76fb8311
I20260619 15:01:27.463133 30731 ts_tablet_manager.cc:1939] T 4bdd7e639f3b426bb3caaf6c76fb8311 P c946147e3d7e4feebea827aef9ecbd63: Deleting consensus metadata
I20260619 15:01:27.464788 30216 catalog_manager.cc:4841] TS c946147e3d7e4feebea827aef9ecbd63 (127.11.175.195:40111): tablet 4bdd7e639f3b426bb3caaf6c76fb8311 (table test-table [id=fe31f885c358402595e91270aa57dc59]) successfully deleted
I20260619 15:01:27.559914 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 30292
W20260619 15:01:27.579275 30218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.175.193:46209 (error 108)
I20260619 15:01:27.580062 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 30426
I20260619 15:01:27.581650 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.581058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:46209 (local address 127.0.0.1:33518)
0619 15:01:27.581315 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.581466 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260619 15:01:27.581913 30218 catalog_manager.cc:4584] TS 9c83e86328f841f78d42d38d9c8114ab (127.11.175.193:46209): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4bdd7e639f3b426bb3caaf6c76fb8311: Network error: Client connection negotiation failed: client connection to 127.11.175.193:46209: connect: Connection refused (error 111)
Jun 19 15:01:27 dist-test-slave-76rk krb5kdc[30186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 15:01:27.598212 30705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.581044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.11.175.194:44301)
0619 15:01:27.581231 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.581238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.581314 (+ 76us) server_negotiation.cc:407] Connection header received
0619 15:01:27.581429 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.581433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.581505 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.581586 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:27.583057 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.583883 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.585352 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.585746 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.585853 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.590753 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:27.590771 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:27.590787 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:27.590832 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:27.594124 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:27.595296 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:27.595305 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:27.595307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:27.595475 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:27.596308 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:27.596313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:27.596315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:27.596854 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:27.597106 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.597693 (+ 587us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.597947 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 15:01:27.599165 30549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:27.635938 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.635387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:46209 (local address 127.0.0.1:33530)
0619 15:01:27.635646 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.635773 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260619 15:01:27.637225 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.636761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:46209 (local address 127.0.0.1:33542)
0619 15:01:27.636943 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.637039 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 15:01:27.691329 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 30426
I20260619 15:01:27.698071 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.697426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:46209 (local address 127.0.0.1:33544)
0619 15:01:27.697692 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.697848 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 15:01:27.699282 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.698877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:46209 (local address 127.0.0.1:33560)
0619 15:01:27.699059 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.699146 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 15:01:27.709779 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 30560
Jun 19 15:01:27 dist-test-slave-76rk krb5kdc[30186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 15:01:27.726788 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.710536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55910 (local address 127.11.175.195:40111)
0619 15:01:27.710753 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:27.710758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:27.710800 (+ 42us) server_negotiation.cc:407] Connection header received
0619 15:01:27.710958 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:27.710962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:27.711027 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:27.711116 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:27.712543 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.713288 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.715252 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:27.715680 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:27.715803 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:27.720419 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:27.720436 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:27.720453 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:27.720494 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:27.723851 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:27.724749 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:27.724758 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:27.724761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:27.724919 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:27.725310 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:27.725313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:27.725316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:27.725686 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:27.725915 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:27.726227 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 15:01:27.726444 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 15:01:27.727464 30683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:27.780213 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.779716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:46209 (local address 127.0.0.1:33568)
0619 15:01:27.779952 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.780046 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260619 15:01:27.781379 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:27.780961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:46209 (local address 127.0.0.1:33582)
0619 15:01:27.781167 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:27.781255 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260619 15:01:27.818727 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 30560
I20260619 15:01:27.836169 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 30199
I20260619 15:01:27.837402 30261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:27.950040 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 30199
2026-06-19T15:01:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4464 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T15:01:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:28Z Disabled control of system clock
I20260619 15:01:28.033331 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:45429
--webserver_interface=127.11.175.254
--webserver_port=0
--builtin_ntp_servers=127.11.175.212:41951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:45429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 15:01:28.379999 30743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:28.380333 30743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:28.380481 30743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:28.388067 30743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:28.388200 30743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:28.388279 30743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:28.388375 30743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:28.388464 30743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:28.398095 30743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:41951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:45429
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:45429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:28.399577 30743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:28.401422 30743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:28.414299 30751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:28.414294 30748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:28.414345 30749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:28.415712 30743 server_base.cc:1034] running on GCE node
I20260619 15:01:28.416728 30743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:28.418308 30743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:28.419575 30743 hybrid_clock.cc:648] HybridClock initialized: now 1781881288419511 us; error 92 us; skew 500 ppm
I20260619 15:01:28.419991 30743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:28.426608 30743 webserver.cc:460] Webserver started at http://127.11.175.254:42593/ using document root <none> and password file <none>
I20260619 15:01:28.427516 30743 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:28.427690 30743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:28.428085 30743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:28.431631 30743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "7d861082bf934dc38e26e7658e4e2b25"
format_stamp: "Formatted at 2026-06-19 15:01:28 on dist-test-slave-76rk"
server_key: "601f5334b776490f4b18068faaabf127"
server_key_iv: "ec3b7e95fba28b9cf18f0dfec014e0ed"
server_key_version: "encryptionkey@0"
I20260619 15:01:28.432593 30743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "7d861082bf934dc38e26e7658e4e2b25"
format_stamp: "Formatted at 2026-06-19 15:01:28 on dist-test-slave-76rk"
server_key: "601f5334b776490f4b18068faaabf127"
server_key_iv: "ec3b7e95fba28b9cf18f0dfec014e0ed"
server_key_version: "encryptionkey@0"
I20260619 15:01:28.438895 30743 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260619 15:01:28.442902 30757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:28.444535 30743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 15:01:28.444819 30743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "7d861082bf934dc38e26e7658e4e2b25"
format_stamp: "Formatted at 2026-06-19 15:01:28 on dist-test-slave-76rk"
server_key: "601f5334b776490f4b18068faaabf127"
server_key_iv: "ec3b7e95fba28b9cf18f0dfec014e0ed"
server_key_version: "encryptionkey@0"
I20260619 15:01:28.445150 30743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:28.460667 30743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:28.461665 30743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:28.462028 30743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:28.484426 30743 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:45429
I20260619 15:01:28.484432 30808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:45429 every 8 connection(s)
I20260619 15:01:28.485994 30743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:28.490000 30809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:28.490800 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 30743
I20260619 15:01:28.490986 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:28.491559 11967 external_mini_cluster.cc:1428] Setting key 4a35791e9d5c632561322ca58081db0d
I20260619 15:01:28.504122 30809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25: Bootstrap starting.
I20260619 15:01:28.506000 30812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:28.493648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33804 (local address 127.11.175.254:45429)
0619 15:01:28.494895 (+ 1247us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:28.494907 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:28.494950 (+ 43us) server_negotiation.cc:407] Connection header received
0619 15:01:28.496966 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:28.496992 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:28.497523 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:28.498000 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:28.498577 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:28.499890 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:28.501363 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:28.502112 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:28.502337 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:28.503576 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:28.503584 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:28.503599 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:28.504193 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:28.504333 (+ 140us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:28.504355 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:28.504480 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:28.504717 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 15:01:28.505039 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":471,"threads_started":1}
I20260619 15:01:28.507865 30809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:28.509209 30809 log.cc:824] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:28.512531 30809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25: No bootstrap required, opened a new log
I20260619 15:01:28.521648 30809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d861082bf934dc38e26e7658e4e2b25" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45429 } }
I20260619 15:01:28.522265 30809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:28.522411 30809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d861082bf934dc38e26e7658e4e2b25, State: Initialized, Role: FOLLOWER
I20260619 15:01:28.523149 30809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d861082bf934dc38e26e7658e4e2b25" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45429 } }
I20260619 15:01:28.523391 30809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:28.523499 30809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:28.523644 30809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:28.526049 30809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d861082bf934dc38e26e7658e4e2b25" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45429 } }
I20260619 15:01:28.526702 30809 leader_election.cc:302] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d861082bf934dc38e26e7658e4e2b25; no voters:
I20260619 15:01:28.527426 30809 leader_election.cc:288] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:28.527648 30814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:28.528993 30814 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [term 1 LEADER]: Becoming Leader. State: Replica: 7d861082bf934dc38e26e7658e4e2b25, State: Running, Role: LEADER
I20260619 15:01:28.529572 30814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d861082bf934dc38e26e7658e4e2b25" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45429 } }
I20260619 15:01:28.530143 30809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:28.533166 30815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d861082bf934dc38e26e7658e4e2b25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d861082bf934dc38e26e7658e4e2b25" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45429 } } }
I20260619 15:01:28.533651 30815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:28.535190 30816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d861082bf934dc38e26e7658e4e2b25. Latest consensus state: current_term: 1 leader_uuid: "7d861082bf934dc38e26e7658e4e2b25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d861082bf934dc38e26e7658e4e2b25" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 45429 } } }
I20260619 15:01:28.535681 30816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:28.536089 30823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:28.542316 30823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:28.553517 30823 catalog_manager.cc:1347] Generated new cluster ID: 0383ce67f894458aaf8980261073b6d5
I20260619 15:01:28.553695 30823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:28.578969 30823 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:28.580350 30823 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:28.589376 30823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25: Generated new TSK 0
I20260619 15:01:28.590277 30823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:28.605057 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45429
--builtin_ntp_servers=127.11.175.212:41951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 15:01:28.979610 30833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:28.979912 30833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:28.980082 30833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:28.987919 30833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:28.988037 30833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:28.988206 30833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 15:01:28.997979 30833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:41951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:28.999383 30833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:29.001209 30833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:29.015146 30839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:29.015192 30841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:29.015858 30838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:29.015990 30833 server_base.cc:1034] running on GCE node
I20260619 15:01:29.016743 30833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:29.017793 30833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:29.019034 30833 hybrid_clock.cc:648] HybridClock initialized: now 1781881289018947 us; error 85 us; skew 500 ppm
I20260619 15:01:29.019397 30833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:29.021904 30833 webserver.cc:460] Webserver started at http://127.11.175.193:33875/ using document root <none> and password file <none>
I20260619 15:01:29.022718 30833 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:29.022863 30833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:29.023187 30833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:29.026188 30833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "874a757c5e3d4cef8903ef99b129fd81"
format_stamp: "Formatted at 2026-06-19 15:01:29 on dist-test-slave-76rk"
server_key: "85fde8f84c63c3fffa1a151917145541"
server_key_iv: "c8d58d9e37ae4d9f7a7f0bb7b74bf5bf"
server_key_version: "encryptionkey@0"
I20260619 15:01:29.027107 30833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "874a757c5e3d4cef8903ef99b129fd81"
format_stamp: "Formatted at 2026-06-19 15:01:29 on dist-test-slave-76rk"
server_key: "85fde8f84c63c3fffa1a151917145541"
server_key_iv: "c8d58d9e37ae4d9f7a7f0bb7b74bf5bf"
server_key_version: "encryptionkey@0"
I20260619 15:01:29.032810 30833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 15:01:29.036701 30847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:29.038205 30833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 15:01:29.038403 30833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "874a757c5e3d4cef8903ef99b129fd81"
format_stamp: "Formatted at 2026-06-19 15:01:29 on dist-test-slave-76rk"
server_key: "85fde8f84c63c3fffa1a151917145541"
server_key_iv: "c8d58d9e37ae4d9f7a7f0bb7b74bf5bf"
server_key_version: "encryptionkey@0"
I20260619 15:01:29.038630 30833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:29.053788 30833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:29.054847 30833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:29.055186 30833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:29.056290 30833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:29.058043 30833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:29.058167 30833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:29.058315 30833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:29.058426 30833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:29.093211 30833 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:39889
I20260619 15:01:29.093310 30959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:39889 every 8 connection(s)
I20260619 15:01:29.094775 30833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 15:01:29.098739 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 30833
I20260619 15:01:29.098945 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 15:01:29.099417 11967 external_mini_cluster.cc:1428] Setting key afd7c2d26649e9d5d0303f333d3e7f6b
I20260619 15:01:29.102454 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45429
--builtin_ntp_servers=127.11.175.212:41951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 15:01:29.109982 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:29.097951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:49349 (local address 127.11.175.254:45429)
0619 15:01:29.098458 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:29.098467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:29.100414 (+ 1947us) server_negotiation.cc:407] Connection header received
0619 15:01:29.102473 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:29.102479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:29.102578 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:29.102805 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:29.104038 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:29.104845 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:29.106617 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:29.107133 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:29.107304 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:29.108978 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:29.108984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:29.108988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:29.109010 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:29.109056 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:29.109068 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:29.109179 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:29.109464 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 15:01:29.109587 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":220,"threads_started":1}
I20260619 15:01:29.111264 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:29.098689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45429 (local address 127.11.175.193:49349)
0619 15:01:29.099977 (+ 1288us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:29.100028 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:29.102135 (+ 2107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:29.103071 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:29.103082 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:29.103190 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:29.103845 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:29.103864 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:29.105034 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:29.105038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:29.106450 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:29.106463 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:29.107414 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:29.107422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:29.107690 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:29.108564 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 15:01:29.108567 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:29.108623 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 15:01:29.108726 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:29.109194 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:29.109209 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:29.109218 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 15:01:29.109451 (+ 233us) client_negotiation.cc:240] Negotiation successful
0619 15:01:29.109955 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":450,"threads_started":1}
I20260619 15:01:29.113059 30960 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45429
I20260619 15:01:29.113437 30960 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:29.114447 30960 heartbeater.cc:511] Master 127.11.175.254:45429 requested a full tablet report, sending...
I20260619 15:01:29.116942 30774 ts_manager.cc:194] Registered new tserver with Master: 874a757c5e3d4cef8903ef99b129fd81 (127.11.175.193:39889)
I20260619 15:01:29.118557 30774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.193:49349
W20260619 15:01:29.449972 30964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:29.450299 30964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:29.450443 30964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:29.457911 30964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:29.458029 30964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:29.458217 30964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 15:01:29.467876 30964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:41951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:29.469201 30964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:29.471060 30964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:29.486354 30969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:29.486274 30970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:29.487197 30964 server_base.cc:1034] running on GCE node
W20260619 15:01:29.487267 30972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:29.487967 30964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:29.489094 30964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:29.490356 30964 hybrid_clock.cc:648] HybridClock initialized: now 1781881289490286 us; error 67 us; skew 500 ppm
I20260619 15:01:29.490806 30964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:29.493368 30964 webserver.cc:460] Webserver started at http://127.11.175.194:42959/ using document root <none> and password file <none>
I20260619 15:01:29.494107 30964 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:29.494195 30964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:29.494549 30964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:29.497588 30964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "c799b06c2c5b4a25b56c0541f0f36ed1"
format_stamp: "Formatted at 2026-06-19 15:01:29 on dist-test-slave-76rk"
server_key: "e295e3f9f9cc85157f8e8f8006e35a46"
server_key_iv: "6ed40643d5c8e0ff43f5476514ec5763"
server_key_version: "encryptionkey@0"
I20260619 15:01:29.498421 30964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "c799b06c2c5b4a25b56c0541f0f36ed1"
format_stamp: "Formatted at 2026-06-19 15:01:29 on dist-test-slave-76rk"
server_key: "e295e3f9f9cc85157f8e8f8006e35a46"
server_key_iv: "6ed40643d5c8e0ff43f5476514ec5763"
server_key_version: "encryptionkey@0"
I20260619 15:01:29.503888 30964 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 15:01:29.507573 30978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:29.509105 30964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 15:01:29.509404 30964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "c799b06c2c5b4a25b56c0541f0f36ed1"
format_stamp: "Formatted at 2026-06-19 15:01:29 on dist-test-slave-76rk"
server_key: "e295e3f9f9cc85157f8e8f8006e35a46"
server_key_iv: "6ed40643d5c8e0ff43f5476514ec5763"
server_key_version: "encryptionkey@0"
I20260619 15:01:29.509650 30964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:29.541198 30964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:29.542090 30964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:29.542418 30964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:29.543471 30964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:29.545003 30964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:29.545138 30964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:29.545280 30964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:29.545367 30964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:29.580639 30964 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:39249
I20260619 15:01:29.580729 31090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:39249 every 8 connection(s)
I20260619 15:01:29.582036 30964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 15:01:29.592243 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 30964
I20260619 15:01:29.592458 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 15:01:29.592963 11967 external_mini_cluster.cc:1428] Setting key c8bfc9d3d3e6af3f55a4a5aa2cc9706c
I20260619 15:01:29.595553 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:29.584785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:49703 (local address 127.11.175.254:45429)
0619 15:01:29.585037 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:29.585043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:29.586538 (+ 1495us) server_negotiation.cc:407] Connection header received
0619 15:01:29.588405 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:29.588409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:29.588485 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:29.588580 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:29.589695 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:29.590559 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:29.592468 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:29.592949 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:29.593100 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:29.594684 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:29.594690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:29.594695 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:29.594718 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:29.594768 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:29.594787 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:29.594880 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:29.595196 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 15:01:29.595305 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 15:01:29.596535 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45429
--builtin_ntp_servers=127.11.175.212:41951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 15:01:29.597301 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:29.585410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45429 (local address 127.11.175.194:49703)
0619 15:01:29.586331 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:29.586368 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:29.588087 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:29.588798 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:29.588806 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:29.588899 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:29.589514 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:29.589533 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:29.590729 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:29.590733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:29.592232 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:29.592249 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:29.593197 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:29.593204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:29.593512 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:29.594313 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 15:01:29.594316 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:29.594370 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 15:01:29.594455 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:29.594920 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:29.594934 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:29.594943 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 15:01:29.595156 (+ 213us) client_negotiation.cc:240] Negotiation successful
0619 15:01:29.595651 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":286,"threads_started":1}
I20260619 15:01:29.599207 31091 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45429
I20260619 15:01:29.599548 31091 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:29.600531 31091 heartbeater.cc:511] Master 127.11.175.254:45429 requested a full tablet report, sending...
I20260619 15:01:29.602231 30774 ts_manager.cc:194] Registered new tserver with Master: c799b06c2c5b4a25b56c0541f0f36ed1 (127.11.175.194:39249)
I20260619 15:01:29.603147 30774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.194:49703
W20260619 15:01:29.942540 31094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:29.942955 31094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:29.943123 31094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:29.950801 31094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:29.950976 31094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:29.951185 31094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 15:01:29.960772 31094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:41951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:45429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:29.962173 31094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:29.964056 31094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:29.979553 31102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:29.979558 31099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:29.979590 31100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:29.980451 31094 server_base.cc:1034] running on GCE node
I20260619 15:01:29.981092 31094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:29.982194 31094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:29.983435 31094 hybrid_clock.cc:648] HybridClock initialized: now 1781881289983358 us; error 66 us; skew 500 ppm
I20260619 15:01:29.983850 31094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:29.986408 31094 webserver.cc:460] Webserver started at http://127.11.175.195:37685/ using document root <none> and password file <none>
I20260619 15:01:29.987210 31094 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:29.987306 31094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:29.987692 31094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:29.991446 31094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "f059fe737f0742dd871e97b491ee677a"
format_stamp: "Formatted at 2026-06-19 15:01:29 on dist-test-slave-76rk"
server_key: "a9d378fa85e98dd0fe6aec28828ee862"
server_key_iv: "1cf5ab8b10688a3f893d646a1bd01f3d"
server_key_version: "encryptionkey@0"
I20260619 15:01:29.992349 31094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "f059fe737f0742dd871e97b491ee677a"
format_stamp: "Formatted at 2026-06-19 15:01:29 on dist-test-slave-76rk"
server_key: "a9d378fa85e98dd0fe6aec28828ee862"
server_key_iv: "1cf5ab8b10688a3f893d646a1bd01f3d"
server_key_version: "encryptionkey@0"
I20260619 15:01:29.998169 31094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 15:01:30.002326 31108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:30.004011 31094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 15:01:30.004257 31094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "f059fe737f0742dd871e97b491ee677a"
format_stamp: "Formatted at 2026-06-19 15:01:29 on dist-test-slave-76rk"
server_key: "a9d378fa85e98dd0fe6aec28828ee862"
server_key_iv: "1cf5ab8b10688a3f893d646a1bd01f3d"
server_key_version: "encryptionkey@0"
I20260619 15:01:30.004503 31094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:30.021955 31094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:30.023234 31094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:30.023648 31094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:30.024834 31094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:30.026454 31094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:30.026583 31094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:30.026746 31094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:30.026841 31094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:30.064059 31094 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:44121
I20260619 15:01:30.064180 31220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:44121 every 8 connection(s)
I20260619 15:01:30.066015 31094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 15:01:30.067124 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 31094
I20260619 15:01:30.067322 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 15:01:30.067863 11967 external_mini_cluster.cc:1428] Setting key 83f952d0afc3a7fad440c602a8a4c248
I20260619 15:01:30.084694 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.069700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:49431 (local address 127.11.175.254:45429)
0619 15:01:30.069942 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.069947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.073345 (+ 3398us) server_negotiation.cc:407] Connection header received
0619 15:01:30.075492 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.075498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.075602 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.075721 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:30.077728 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.078516 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.080599 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.081036 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.081190 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.083107 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:30.083113 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:30.083116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:30.083143 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:30.083179 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:30.083192 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:30.083351 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.084016 (+ 665us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.084202 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 15:01:30.086267 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.071066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:45429 (local address 127.11.175.195:49431)
0619 15:01:30.073104 (+ 2038us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:30.073150 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:30.075145 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:30.076052 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:30.076071 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:30.076213 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:30.077485 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.077510 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.078765 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.078769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.080408 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.080425 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.081308 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.081315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.081583 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.082581 (+ 998us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 15:01:30.082584 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:30.082717 (+ 133us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 15:01:30.082840 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:30.083350 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:30.083362 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:30.083374 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 15:01:30.083610 (+ 236us) client_negotiation.cc:240] Negotiation successful
0619 15:01:30.084147 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":416,"threads_started":1}
I20260619 15:01:30.088492 31221 heartbeater.cc:347] Connected to a master server at 127.11.175.254:45429
I20260619 15:01:30.088817 31221 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:30.089586 31221 heartbeater.cc:511] Master 127.11.175.254:45429 requested a full tablet report, sending...
I20260619 15:01:30.091195 30774 ts_manager.cc:194] Registered new tserver with Master: f059fe737f0742dd871e97b491ee677a (127.11.175.195:44121)
I20260619 15:01:30.092046 30774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.195:49431
I20260619 15:01:30.100544 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 15:01:30.112092 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.105410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33814 (local address 127.11.175.254:45429)
0619 15:01:30.105774 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.105780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.105961 (+ 181us) server_negotiation.cc:407] Connection header received
0619 15:01:30.106100 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.106106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.106180 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.106277 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:30.106925 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.107620 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.109308 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.109833 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.109940 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.111253 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:30.111259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:30.111262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:30.111284 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:30.111318 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:30.111330 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:30.111450 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.111642 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.111778 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260619 15:01:30.123023 30960 heartbeater.cc:503] Master 127.11.175.254:45429 was elected leader, sending a full tablet report...
I20260619 15:01:30.117344 30774 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 15:01:30.124910 30774 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 15:01:30.161182 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.140377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45734 (local address 127.11.175.194:39249)
0619 15:01:30.153568 (+ 13191us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.153577 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.153606 (+ 29us) server_negotiation.cc:407] Connection header received
0619 15:01:30.153703 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.153709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.153883 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.154098 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:30.154610 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.155630 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.158600 (+ 2970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.159942 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.160125 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.160386 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.160638 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.160818 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13040,"thread_start_us":12921,"threads_started":1}
I20260619 15:01:30.161235 31233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.139579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:39249 (local address 127.0.0.1:45734)
0619 15:01:30.151387 (+ 11808us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:30.151409 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:30.151558 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:30.154078 (+ 2520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:30.154083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:30.154110 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:30.154436 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.154445 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.155791 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.155798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.158329 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.158345 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.160154 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.160160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.160365 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.160419 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 15:01:30.160580 (+ 161us) client_negotiation.cc:240] Negotiation successful
0619 15:01:30.160807 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11669,"mutex_wait_us":2,"thread_start_us":11972,"threads_started":1}
I20260619 15:01:30.162583 31235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.139149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:39889 (local address 127.0.0.1:35776)
0619 15:01:30.151050 (+ 11901us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:30.151105 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:30.151350 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:30.152778 (+ 1428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:30.152787 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:30.152828 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:30.153443 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.153465 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.156270 (+ 2805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.156275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.158342 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.158351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.161237 (+ 2886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.161248 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.161615 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.161684 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 15:01:30.161782 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 15:01:30.162001 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11458,"mutex_wait_us":1,"thread_start_us":11938,"threads_started":1}
I20260619 15:01:30.164448 31234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.139459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35776 (local address 127.11.175.193:39889)
0619 15:01:30.151035 (+ 11576us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.151057 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.151916 (+ 859us) server_negotiation.cc:407] Connection header received
0619 15:01:30.152076 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.152100 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.152477 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.152746 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:30.153763 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.156083 (+ 2320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.158572 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.160867 (+ 2295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.163049 (+ 2182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.163461 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.163585 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.163891 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11356,"thread_start_us":11188,"threads_started":1}
I20260619 15:01:30.166271 31026 tablet_service.cc:1467] Processing CreateTablet for tablet 8977c420e2264cd1b4a28e689e7b8236 (DEFAULT_TABLE table=test-table [id=fd793c57e0c8414db4e02d151b32e747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:30.167990 31026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8977c420e2264cd1b4a28e689e7b8236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:30.169214 30895 tablet_service.cc:1467] Processing CreateTablet for tablet 8977c420e2264cd1b4a28e689e7b8236 (DEFAULT_TABLE table=test-table [id=fd793c57e0c8414db4e02d151b32e747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:30.170742 30895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8977c420e2264cd1b4a28e689e7b8236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:30.174311 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.158458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:44121 (local address 127.0.0.1:54066)
0619 15:01:30.159057 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:30.159077 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:30.159199 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:30.166137 (+ 6938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:30.166144 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:30.166180 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:30.166559 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.166573 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.168638 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.168644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.170763 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.170788 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.173439 (+ 2651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.173447 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.173645 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.173689 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 15:01:30.173799 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 15:01:30.173962 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":631,"threads_started":1}
I20260619 15:01:30.175037 31237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.152553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.11.175.195:44121)
0619 15:01:30.165499 (+ 12946us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.165506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.165540 (+ 34us) server_negotiation.cc:407] Connection header received
0619 15:01:30.165633 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.165639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.165847 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.166126 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:30.166793 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.168427 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.171109 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.172816 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.173063 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.173285 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.174424 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.174659 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12773,"thread_start_us":13816,"threads_started":1}
I20260619 15:01:30.179488 31156 tablet_service.cc:1467] Processing CreateTablet for tablet 8977c420e2264cd1b4a28e689e7b8236 (DEFAULT_TABLE table=test-table [id=fd793c57e0c8414db4e02d151b32e747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:30.181123 31156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8977c420e2264cd1b4a28e689e7b8236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:30.184461 31240 tablet_bootstrap.cc:492] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: Bootstrap starting.
I20260619 15:01:30.187451 31239 tablet_bootstrap.cc:492] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: Bootstrap starting.
I20260619 15:01:30.188302 31240 tablet_bootstrap.cc:654] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:30.189817 31240 log.cc:824] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:30.191330 31239 tablet_bootstrap.cc:654] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:30.192715 31239 log.cc:824] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:30.193586 31240 tablet_bootstrap.cc:492] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: No bootstrap required, opened a new log
I20260619 15:01:30.194671 31240 ts_tablet_manager.cc:1397] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260619 15:01:30.196249 31243 tablet_bootstrap.cc:492] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: Bootstrap starting.
I20260619 15:01:30.199811 31243 tablet_bootstrap.cc:654] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:30.200899 31243 log.cc:824] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:30.201777 31239 tablet_bootstrap.cc:492] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: No bootstrap required, opened a new log
I20260619 15:01:30.202149 31239 ts_tablet_manager.cc:1397] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.005s
I20260619 15:01:30.204107 31243 tablet_bootstrap.cc:492] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: No bootstrap required, opened a new log
I20260619 15:01:30.204401 31243 ts_tablet_manager.cc:1397] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 15:01:30.207024 31240 raft_consensus.cc:348] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874a757c5e3d4cef8903ef99b129fd81" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39889 } } peers { permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 } } peers { permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 } }
I20260619 15:01:30.207616 31240 raft_consensus.cc:374] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:30.207747 31240 raft_consensus.cc:729] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874a757c5e3d4cef8903ef99b129fd81, State: Initialized, Role: FOLLOWER
I20260619 15:01:30.208609 31240 consensus_queue.cc:260] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874a757c5e3d4cef8903ef99b129fd81" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39889 } } peers { permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 } } peers { permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 } }
I20260619 15:01:30.210785 31240 ts_tablet_manager.cc:1428] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260619 15:01:30.212976 31239 raft_consensus.cc:348] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874a757c5e3d4cef8903ef99b129fd81" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39889 } } peers { permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 } } peers { permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 } }
I20260619 15:01:30.213447 31239 raft_consensus.cc:374] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:30.213583 31239 raft_consensus.cc:729] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c799b06c2c5b4a25b56c0541f0f36ed1, State: Initialized, Role: FOLLOWER
I20260619 15:01:30.214306 31239 consensus_queue.cc:260] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874a757c5e3d4cef8903ef99b129fd81" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39889 } } peers { permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 } } peers { permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 } }
I20260619 15:01:30.215536 31091 heartbeater.cc:503] Master 127.11.175.254:45429 was elected leader, sending a full tablet report...
I20260619 15:01:30.215607 31243 raft_consensus.cc:348] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874a757c5e3d4cef8903ef99b129fd81" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39889 } } peers { permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 } } peers { permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 } }
I20260619 15:01:30.216080 31243 raft_consensus.cc:374] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:30.216177 31243 raft_consensus.cc:729] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f059fe737f0742dd871e97b491ee677a, State: Initialized, Role: FOLLOWER
I20260619 15:01:30.216290 31239 ts_tablet_manager.cc:1428] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260619 15:01:30.217006 31243 consensus_queue.cc:260] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874a757c5e3d4cef8903ef99b129fd81" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39889 } } peers { permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 } } peers { permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 } }
I20260619 15:01:30.218325 31221 heartbeater.cc:503] Master 127.11.175.254:45429 was elected leader, sending a full tablet report...
I20260619 15:01:30.219136 31243 ts_tablet_manager.cc:1428] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260619 15:01:30.292254 31245 raft_consensus.cc:482] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 15:01:30.292598 31245 raft_consensus.cc:504] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874a757c5e3d4cef8903ef99b129fd81" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39889 } } peers { permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 } } peers { permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 } }
I20260619 15:01:30.294536 31245 leader_election.cc:288] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f059fe737f0742dd871e97b491ee677a (127.11.175.195:44121), c799b06c2c5b4a25b56c0541f0f36ed1 (127.11.175.194:39249)
I20260619 15:01:30.302881 31237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.295138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:32997 (local address 127.11.175.195:44121)
0619 15:01:30.295349 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.295357 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.296019 (+ 662us) server_negotiation.cc:407] Connection header received
0619 15:01:30.296148 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.296152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.296229 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.296334 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:30.297269 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.298427 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.300974 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.301905 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.302079 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.302201 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.302468 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.302611 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 15:01:30.303026 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.295056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:33129 (local address 127.11.175.194:39249)
0619 15:01:30.295402 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.295409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.295897 (+ 488us) server_negotiation.cc:407] Connection header received
0619 15:01:30.296138 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.296143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.296229 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.296335 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:30.297271 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.298447 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.301057 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.301930 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.302117 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.302190 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.302559 (+ 369us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.302747 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260619 15:01:30.303148 31249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.294951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:44121 (local address 127.11.175.193:32997)
0619 15:01:30.295922 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:30.295938 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:30.296049 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:30.296378 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:30.296384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:30.296443 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:30.297038 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.297046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.298614 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.298621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.300749 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.300768 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.302114 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.302121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.302296 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.302321 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 15:01:30.302434 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 15:01:30.302577 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":194,"spinlock_wait_cycles":9856,"thread_start_us":621,"threads_started":1}
I20260619 15:01:30.303171 31248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.294791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:39249 (local address 127.11.175.193:33129)
0619 15:01:30.295757 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:30.295778 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:30.295968 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:30.296378 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:30.296384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:30.296442 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:30.296993 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.297015 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.298690 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.298696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.300757 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:30.300772 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.302167 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.302174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:30.302370 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.302409 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 15:01:30.302523 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 15:01:30.302684 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":99,"spinlock_wait_cycles":34944,"thread_start_us":546,"threads_started":1}
I20260619 15:01:30.304399 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8977c420e2264cd1b4a28e689e7b8236" candidate_uuid: "874a757c5e3d4cef8903ef99b129fd81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" is_pre_election: true
I20260619 15:01:30.304400 31176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8977c420e2264cd1b4a28e689e7b8236" candidate_uuid: "874a757c5e3d4cef8903ef99b129fd81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f059fe737f0742dd871e97b491ee677a" is_pre_election: true
I20260619 15:01:30.305004 31046 raft_consensus.cc:2454] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 874a757c5e3d4cef8903ef99b129fd81 in term 0.
I20260619 15:01:30.305004 31176 raft_consensus.cc:2454] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 874a757c5e3d4cef8903ef99b129fd81 in term 0.
I20260619 15:01:30.305824 30850 leader_election.cc:302] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 874a757c5e3d4cef8903ef99b129fd81, c799b06c2c5b4a25b56c0541f0f36ed1, f059fe737f0742dd871e97b491ee677a; no voters:
I20260619 15:01:30.306337 31245 raft_consensus.cc:2781] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 15:01:30.306481 31245 raft_consensus.cc:482] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 15:01:30.306702 31245 raft_consensus.cc:3037] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:30.309039 31245 raft_consensus.cc:504] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874a757c5e3d4cef8903ef99b129fd81" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39889 } } peers { permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 } } peers { permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 } }
I20260619 15:01:30.309798 31245 leader_election.cc:288] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [CANDIDATE]: Term 1 election: Requested vote from peers f059fe737f0742dd871e97b491ee677a (127.11.175.195:44121), c799b06c2c5b4a25b56c0541f0f36ed1 (127.11.175.194:39249)
I20260619 15:01:30.310278 31176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8977c420e2264cd1b4a28e689e7b8236" candidate_uuid: "874a757c5e3d4cef8903ef99b129fd81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f059fe737f0742dd871e97b491ee677a"
I20260619 15:01:30.310323 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8977c420e2264cd1b4a28e689e7b8236" candidate_uuid: "874a757c5e3d4cef8903ef99b129fd81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1"
I20260619 15:01:30.310509 31176 raft_consensus.cc:3037] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:30.310531 31046 raft_consensus.cc:3037] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:30.312863 31176 raft_consensus.cc:2454] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874a757c5e3d4cef8903ef99b129fd81 in term 1.
I20260619 15:01:30.312881 31046 raft_consensus.cc:2454] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874a757c5e3d4cef8903ef99b129fd81 in term 1.
I20260619 15:01:30.313468 30848 leader_election.cc:302] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 874a757c5e3d4cef8903ef99b129fd81, c799b06c2c5b4a25b56c0541f0f36ed1; no voters:
I20260619 15:01:30.313827 31245 raft_consensus.cc:2781] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:30.314520 31245 raft_consensus.cc:686] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 1 LEADER]: Becoming Leader. State: Replica: 874a757c5e3d4cef8903ef99b129fd81, State: Running, Role: LEADER
I20260619 15:01:30.315042 31245 consensus_queue.cc:237] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874a757c5e3d4cef8903ef99b129fd81" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39889 } } peers { permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 } } peers { permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 } }
W20260619 15:01:30.317992 31222 tablet.cc:2369] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:01:30.320055 30773 catalog_manager.cc:5495] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 874a757c5e3d4cef8903ef99b129fd81 (127.11.175.193). New cstate: current_term: 1 leader_uuid: "874a757c5e3d4cef8903ef99b129fd81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874a757c5e3d4cef8903ef99b129fd81" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 39889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 } health_report { overall_health: UNKNOWN } } }
W20260619 15:01:30.334420 31092 tablet.cc:2369] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 15:01:30.347863 30961 tablet.cc:2369] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:01:30.349932 31234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.344169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.11.175.193:39889)
0619 15:01:30.344389 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.344394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.344411 (+ 17us) server_negotiation.cc:407] Connection header received
0619 15:01:30.344510 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.344514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.344597 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.344723 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:01:30.345128 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.345849 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.347383 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.347834 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.347997 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.348649 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:01:30.349259 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:01:30.349462 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.349607 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.349654 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260619 15:01:30.359211 31176 raft_consensus.cc:1261] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a [term 1 FOLLOWER]: Refusing update from remote peer 874a757c5e3d4cef8903ef99b129fd81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:30.359225 31046 raft_consensus.cc:1261] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1 [term 1 FOLLOWER]: Refusing update from remote peer 874a757c5e3d4cef8903ef99b129fd81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:30.360451 31250 consensus_queue.cc:1035] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c799b06c2c5b4a25b56c0541f0f36ed1" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 39249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:30.360867 31245 consensus_queue.cc:1035] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f059fe737f0742dd871e97b491ee677a" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 44121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:30.373750 31255 mvcc.cc:205] Tried to move back new op lower bound from 7298585765299384320 to 7298585765134319616. Current Snapshot: MvccSnapshot[applied={T|T < 7298585765299384320}]
I20260619 15:01:30.376854 31253 mvcc.cc:205] Tried to move back new op lower bound from 7298585765299384320 to 7298585765134319616. Current Snapshot: MvccSnapshot[applied={T|T < 7298585765299384320}]
I20260619 15:01:30.389918 30773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:30.390226 30773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:30.394155 30773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d861082bf934dc38e26e7658e4e2b25: Sending DeleteTablet for 3 replicas of tablet 8977c420e2264cd1b4a28e689e7b8236
I20260619 15:01:30.395561 31156 tablet_service.cc:1514] Processing DeleteTablet for tablet 8977c420e2264cd1b4a28e689e7b8236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:30 UTC) from {username='slave'} at 127.0.0.1:54066
I20260619 15:01:30.395561 31026 tablet_service.cc:1514] Processing DeleteTablet for tablet 8977c420e2264cd1b4a28e689e7b8236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:30 UTC) from {username='slave'} at 127.0.0.1:45734
I20260619 15:01:30.395593 30895 tablet_service.cc:1514] Processing DeleteTablet for tablet 8977c420e2264cd1b4a28e689e7b8236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:30 UTC) from {username='slave'} at 127.0.0.1:35776
I20260619 15:01:30.406527 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 30833
I20260619 15:01:30.406827 31262 tablet_replica.cc:330] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: stopping tablet replica
I20260619 15:01:30.407339 31262 raft_consensus.cc:2229] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:30.407912 31262 raft_consensus.cc:2258] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:30.411023 31262 ts_tablet_manager.cc:1905] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:30.414065 31263 tablet_replica.cc:330] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: stopping tablet replica
I20260619 15:01:30.415100 31264 tablet_replica.cc:330] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: stopping tablet replica
I20260619 15:01:30.415910 31263 raft_consensus.cc:2229] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:30.416642 31264 raft_consensus.cc:2229] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 1 LEADER]: Raft consensus shutting down.
I20260619 15:01:30.417567 31263 raft_consensus.cc:2258] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:30.418499 31264 raft_consensus.cc:2258] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:30.421209 31263 ts_tablet_manager.cc:1905] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:30.421212 31264 ts_tablet_manager.cc:1905] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:30.421389 31262 ts_tablet_manager.cc:1918] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:30.421660 31262 log.cc:1196] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/8977c420e2264cd1b4a28e689e7b8236
I20260619 15:01:30.422303 31262 ts_tablet_manager.cc:1939] T 8977c420e2264cd1b4a28e689e7b8236 P f059fe737f0742dd871e97b491ee677a: Deleting consensus metadata
I20260619 15:01:30.424515 30760 catalog_manager.cc:4841] TS f059fe737f0742dd871e97b491ee677a (127.11.175.195:44121): tablet 8977c420e2264cd1b4a28e689e7b8236 (table test-table [id=fd793c57e0c8414db4e02d151b32e747]) successfully deleted
I20260619 15:01:30.429296 31264 ts_tablet_manager.cc:1918] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:30.429592 31264 log.cc:1196] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/8977c420e2264cd1b4a28e689e7b8236
I20260619 15:01:30.429795 31234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.407911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.11.175.193:39889)
0619 15:01:30.408130 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.408137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.418497 (+ 10360us) server_negotiation.cc:407] Connection header received
0619 15:01:30.419090 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.419096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.419212 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.419297 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:30.422106 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.423438 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.425541 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.426193 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.426302 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.427919 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:30.427931 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:30.427949 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:30.428780 (+ 831us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:30.428844 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:30.428869 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:30.428958 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.429303 (+ 345us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.429412 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 15:01:30.430306 31264 ts_tablet_manager.cc:1939] T 8977c420e2264cd1b4a28e689e7b8236 P 874a757c5e3d4cef8903ef99b129fd81: Deleting consensus metadata
I20260619 15:01:30.431288 30955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:30.434125 31263 ts_tablet_manager.cc:1918] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:30.434428 31263 log.cc:1196] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/8977c420e2264cd1b4a28e689e7b8236
I20260619 15:01:30.435125 31263 ts_tablet_manager.cc:1939] T 8977c420e2264cd1b4a28e689e7b8236 P c799b06c2c5b4a25b56c0541f0f36ed1: Deleting consensus metadata
I20260619 15:01:30.437244 30758 catalog_manager.cc:4841] TS c799b06c2c5b4a25b56c0541f0f36ed1 (127.11.175.194:39249): tablet 8977c420e2264cd1b4a28e689e7b8236 (table test-table [id=fd793c57e0c8414db4e02d151b32e747]) successfully deleted
I20260619 15:01:30.543488 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 30833
I20260619 15:01:30.545255 30760 catalog_manager.cc:4841] TS 874a757c5e3d4cef8903ef99b129fd81 (127.11.175.193:39889): tablet 8977c420e2264cd1b4a28e689e7b8236 (table test-table [id=fd793c57e0c8414db4e02d151b32e747]) successfully deleted
I20260619 15:01:30.562070 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 30964
I20260619 15:01:30.569761 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.562957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45742 (local address 127.11.175.194:39249)
0619 15:01:30.563129 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.563134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.563153 (+ 19us) server_negotiation.cc:407] Connection header received
0619 15:01:30.563318 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.563323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.563403 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.563495 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:30.563915 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.564644 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.565992 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.566433 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.566533 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.568019 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:30.568028 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:30.568043 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:30.568925 (+ 882us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:30.568978 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:30.568999 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:30.569096 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.569372 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.569473 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 15:01:30.570514 31086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:30.661056 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 30964
I20260619 15:01:30.679141 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 31094
I20260619 15:01:30.686542 31237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:30.679981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.11.175.195:44121)
0619 15:01:30.680173 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:30.680180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:30.680203 (+ 23us) server_negotiation.cc:407] Connection header received
0619 15:01:30.680400 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:30.680405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:30.680492 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:30.680595 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:30.680976 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.681782 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.683371 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:30.683772 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:30.683877 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:30.685056 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:30.685065 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:30.685080 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:30.685702 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:30.685752 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:30.685773 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:30.685867 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:30.686051 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 15:01:30.686235 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 15:01:30.687394 31216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:30.777688 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 31094
I20260619 15:01:30.795032 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 30743
I20260619 15:01:30.796268 30804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:30.915127 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 30743
2026-06-19T15:01:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2964 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:31 dist-test-slave-76rk krb5kdc[31272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:31 dist-test-slave-76rk krb5kdc[31272](info): set up 2 sockets
Jun 19 15:01:31 dist-test-slave-76rk krb5kdc[31272](info): commencing operation
krb5kdc: starting...
W20260619 15:01:32.043038 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.006s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:32 dist-test-slave-76rk krb5kdc[31272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:01:32.342875 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:35559
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:35559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:32.698185 31285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:32.698522 31285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:32.698617 31285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:32.706292 31285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:32.706429 31285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:32.706478 31285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:32.706532 31285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:32.706576 31285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:32.717203 31285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:35559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:35559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:32.718667 31285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:32.720058 31285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:32.732635 31293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:32.732615 31291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:32.733268 31290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:32.733700 31285 server_base.cc:1034] running on GCE node
I20260619 15:01:32.734560 31285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:32.736178 31285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:32.737418 31285 hybrid_clock.cc:648] HybridClock initialized: now 1781881292737367 us; error 33 us; skew 500 ppm
I20260619 15:01:32.737871 31285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:32 dist-test-slave-76rk krb5kdc[31272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881292, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:32.741293 31285 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:01:32.743153 31285 webserver.cc:460] Webserver started at http://127.11.175.254:34095/ using document root <none> and password file <none>
I20260619 15:01:32.743903 31285 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:32.744061 31285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:32.744381 31285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:32.747753 31285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "e51c0394ca6c4afeb2a179897e4f3380"
format_stamp: "Formatted at 2026-06-19 15:01:32 on dist-test-slave-76rk"
server_key: "594d8e0e34294e99884726e294716288"
server_key_iv: "14127d1d877586637ad2e3125ef84bee"
server_key_version: "encryptionkey@0"
I20260619 15:01:32.748575 31285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "e51c0394ca6c4afeb2a179897e4f3380"
format_stamp: "Formatted at 2026-06-19 15:01:32 on dist-test-slave-76rk"
server_key: "594d8e0e34294e99884726e294716288"
server_key_iv: "14127d1d877586637ad2e3125ef84bee"
server_key_version: "encryptionkey@0"
I20260619 15:01:32.754333 31285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 15:01:32.758500 31300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:32.760208 31285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 15:01:32.760486 31285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "e51c0394ca6c4afeb2a179897e4f3380"
format_stamp: "Formatted at 2026-06-19 15:01:32 on dist-test-slave-76rk"
server_key: "594d8e0e34294e99884726e294716288"
server_key_iv: "14127d1d877586637ad2e3125ef84bee"
server_key_version: "encryptionkey@0"
I20260619 15:01:32.760771 31285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:32.798907 31285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:32.803678 31285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:32.804003 31285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:32.824442 31285 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:35559
I20260619 15:01:32.824445 31351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:35559 every 8 connection(s)
I20260619 15:01:32.826000 31285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:32.830034 31352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:32.831802 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 31285
I20260619 15:01:32.831998 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:32.832476 11967 external_mini_cluster.cc:1428] Setting key 7367a4241e0364b3a26d0cc8be5b48a2
I20260619 15:01:32.845674 31352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380: Bootstrap starting.
Jun 19 15:01:32 dist-test-slave-76rk krb5kdc[31272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:32.849470 31352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:32.850760 31352 log.cc:824] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:32.854039 31352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380: No bootstrap required, opened a new log
I20260619 15:01:32.858860 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:32.834821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41088 (local address 127.11.175.254:35559)
0619 15:01:32.835738 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:32.835759 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:32.835815 (+ 56us) server_negotiation.cc:407] Connection header received
0619 15:01:32.837650 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:32.837684 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:32.838338 (+ 654us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:32.838841 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:32.840517 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:32.842687 (+ 2170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:32.844098 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:32.845163 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:32.845374 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:32.850326 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:32.850338 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:32.850360 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:32.850421 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:32.853806 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:32.854764 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:32.854774 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:32.854788 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:32.854973 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:32.855397 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:32.855402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:32.855407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:32.856212 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:32.856585 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:32.857190 (+ 605us) server_negotiation.cc:299] Negotiation successful
0619 15:01:32.857655 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":407,"threads_started":1}
I20260619 15:01:32.863991 31352 raft_consensus.cc:348] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51c0394ca6c4afeb2a179897e4f3380" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35559 } }
I20260619 15:01:32.864534 31352 raft_consensus.cc:374] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:32.864651 31352 raft_consensus.cc:729] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51c0394ca6c4afeb2a179897e4f3380, State: Initialized, Role: FOLLOWER
I20260619 15:01:32.865316 31352 consensus_queue.cc:260] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51c0394ca6c4afeb2a179897e4f3380" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35559 } }
I20260619 15:01:32.865585 31352 raft_consensus.cc:388] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:32.865741 31352 raft_consensus.cc:482] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:32.865901 31352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:32.869031 31352 raft_consensus.cc:504] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51c0394ca6c4afeb2a179897e4f3380" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35559 } }
I20260619 15:01:32.869675 31352 leader_election.cc:302] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e51c0394ca6c4afeb2a179897e4f3380; no voters:
I20260619 15:01:32.870361 31352 leader_election.cc:288] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:32.870493 31357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:32.875840 31357 raft_consensus.cc:686] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [term 1 LEADER]: Becoming Leader. State: Replica: e51c0394ca6c4afeb2a179897e4f3380, State: Running, Role: LEADER
I20260619 15:01:32.876789 31352 sys_catalog.cc:564] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:32.876667 31357 consensus_queue.cc:237] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51c0394ca6c4afeb2a179897e4f3380" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35559 } }
I20260619 15:01:32.884974 31359 sys_catalog.cc:455] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e51c0394ca6c4afeb2a179897e4f3380. Latest consensus state: current_term: 1 leader_uuid: "e51c0394ca6c4afeb2a179897e4f3380" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51c0394ca6c4afeb2a179897e4f3380" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35559 } } }
I20260619 15:01:32.885423 31359 sys_catalog.cc:458] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:32.884620 31358 sys_catalog.cc:455] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e51c0394ca6c4afeb2a179897e4f3380" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51c0394ca6c4afeb2a179897e4f3380" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 35559 } } }
I20260619 15:01:32.886142 31358 sys_catalog.cc:458] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:32.887714 31370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:32.892675 31370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:32.902041 31370 catalog_manager.cc:1347] Generated new cluster ID: e64687bbe1674cc4a6f06e18ecdf8516
I20260619 15:01:32.902184 31370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:32.918563 31370 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:32.920035 31370 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:32.928109 31370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380: Generated new TSK 0
I20260619 15:01:32.929114 31370 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.175.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
Entry for principal kudu/127.11.175.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab.
I20260619 15:01:33.009989 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35559
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:33.372821 31378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:33.373172 31378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:33.373343 31378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:33.381511 31378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:33.381677 31378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:33.381855 31378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 15:01:33.392704 31378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.193.keytab
--principal=kudu/127.11.175.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:33.394074 31378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:33.395555 31378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:33.409703 31383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:33.409588 31386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:33.409773 31384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:33.410377 31378 server_base.cc:1034] running on GCE node
I20260619 15:01:33.411136 31378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:33.412310 31378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:33.413579 31378 hybrid_clock.cc:648] HybridClock initialized: now 1781881293413524 us; error 72 us; skew 500 ppm
I20260619 15:01:33.413978 31378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:33 dist-test-slave-76rk krb5kdc[31272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881293, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:33.417527 31378 init.cc:377] Logged in from keytab as kudu/127.11.175.193@KRBTEST.COM (short username kudu)
I20260619 15:01:33.419502 31378 webserver.cc:460] Webserver started at http://127.11.175.193:40355/ using document root <none> and password file <none>
I20260619 15:01:33.420244 31378 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:33.420352 31378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:33.420614 31378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:33.423862 31378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "a5ecda40d4644f27b44fea8f27936177"
format_stamp: "Formatted at 2026-06-19 15:01:33 on dist-test-slave-76rk"
server_key: "de9d55fdf8c28e29cfd6874acea72f94"
server_key_iv: "5fe25756df73695c2fc6c6be842864aa"
server_key_version: "encryptionkey@0"
I20260619 15:01:33.424695 31378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "a5ecda40d4644f27b44fea8f27936177"
format_stamp: "Formatted at 2026-06-19 15:01:33 on dist-test-slave-76rk"
server_key: "de9d55fdf8c28e29cfd6874acea72f94"
server_key_iv: "5fe25756df73695c2fc6c6be842864aa"
server_key_version: "encryptionkey@0"
I20260619 15:01:33.430573 31378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260619 15:01:33.434528 31393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:33.436455 31378 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:01:33.436769 31378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "a5ecda40d4644f27b44fea8f27936177"
format_stamp: "Formatted at 2026-06-19 15:01:33 on dist-test-slave-76rk"
server_key: "de9d55fdf8c28e29cfd6874acea72f94"
server_key_iv: "5fe25756df73695c2fc6c6be842864aa"
server_key_version: "encryptionkey@0"
I20260619 15:01:33.437017 31378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:33.473281 31378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:33.478372 31378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:33.478682 31378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:33.479733 31378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:33.481177 31378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:33.481251 31378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:33.481334 31378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:33.481415 31378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:33.518610 31378 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:44585
I20260619 15:01:33.518755 31505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:44585 every 8 connection(s)
I20260619 15:01:33.520334 31378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 15:01:33.522166 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 31378
I20260619 15:01:33.522341 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 15:01:33.522831 11967 external_mini_cluster.cc:1428] Setting key f4b77fd7d2e8a403e5fcad60e48d05be
Jun 19 15:01:33 dist-test-slave-76rk krb5kdc[31272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881293, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.193@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:33.544675 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:33.523211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:33381 (local address 127.11.175.254:35559)
0619 15:01:33.523732 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:33.523740 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:33.524858 (+ 1118us) server_negotiation.cc:407] Connection header received
0619 15:01:33.527110 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:33.527119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:33.527258 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:33.527371 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:33.529097 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:33.530265 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:33.532030 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:33.532781 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:33.532929 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:33.537572 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:33.537578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:33.537581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:33.537623 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:33.540675 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:33.541802 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:33.541809 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:33.541813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:33.542005 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:33.542613 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:33.542617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:33.542619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:33.543000 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:33.543134 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:33.544174 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0619 15:01:33.544407 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":245,"threads_started":1}
I20260619 15:01:33.546146 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:33.523831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35559 (local address 127.11.175.193:33381)
0619 15:01:33.524666 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:33.524732 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:33.526715 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:33.527583 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:33.527591 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:33.528318 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:33.528905 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:33.528923 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:33.530418 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:33.530422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:33.531874 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:33.531887 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:33.533009 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:33.533014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:33.533228 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:33.533993 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:01:33.534032 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:33.537301 (+ 3269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:33.540846 (+ 3545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:33.540858 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:33.540888 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:33.541616 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:33.542151 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:33.542158 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:33.542165 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:33.542441 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:33.543147 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:33.543158 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:33.543679 (+ 521us) client_negotiation.cc:769] Sending connection context
0619 15:01:33.544004 (+ 325us) client_negotiation.cc:240] Negotiation successful
0619 15:01:33.544609 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":306,"threads_started":1}
I20260619 15:01:33.547995 31506 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35559
I20260619 15:01:33.548403 31506 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:33.549177 31506 heartbeater.cc:511] Master 127.11.175.254:35559 requested a full tablet report, sending...
I20260619 15:01:33.551458 31317 ts_manager.cc:194] Registered new tserver with Master: a5ecda40d4644f27b44fea8f27936177 (127.11.175.193:44585)
WARNING: no policy specified for kudu/127.11.175.194@KRBTEST.COM; defaulting to no policy
I20260619 15:01:33.553066 31317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.193@KRBTEST.COM'} at 127.11.175.193:33381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
Entry for principal kudu/127.11.175.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab.
I20260619 15:01:33.587597 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35559
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:33.940165 31512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:33.940538 31512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:33.940721 31512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:33.948704 31512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:33.948870 31512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:33.949050 31512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 15:01:33.959247 31512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.194.keytab
--principal=kudu/127.11.175.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:33.960736 31512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:33.962108 31512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:33.976428 31517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:33.976354 31518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:33.976461 31520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:33.977480 31512 server_base.cc:1034] running on GCE node
I20260619 15:01:33.978447 31512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:33.979615 31512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:33.980917 31512 hybrid_clock.cc:648] HybridClock initialized: now 1781881293980834 us; error 66 us; skew 500 ppm
I20260619 15:01:33.981338 31512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:33 dist-test-slave-76rk krb5kdc[31272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881293, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:33.984963 31512 init.cc:377] Logged in from keytab as kudu/127.11.175.194@KRBTEST.COM (short username kudu)
I20260619 15:01:33.987035 31512 webserver.cc:460] Webserver started at http://127.11.175.194:34039/ using document root <none> and password file <none>
I20260619 15:01:33.987856 31512 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:33.987970 31512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:33.988310 31512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:33.991621 31512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "59af68efef5b4b44afa2de222e61d1d8"
format_stamp: "Formatted at 2026-06-19 15:01:33 on dist-test-slave-76rk"
server_key: "6da14c5ff5e582a47038e682d21e4ba4"
server_key_iv: "f740766f3a5166d85c9138f80c21f4e7"
server_key_version: "encryptionkey@0"
I20260619 15:01:33.992486 31512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "59af68efef5b4b44afa2de222e61d1d8"
format_stamp: "Formatted at 2026-06-19 15:01:33 on dist-test-slave-76rk"
server_key: "6da14c5ff5e582a47038e682d21e4ba4"
server_key_iv: "f740766f3a5166d85c9138f80c21f4e7"
server_key_version: "encryptionkey@0"
I20260619 15:01:33.998417 31512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 15:01:34.002514 31527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:34.004220 31512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 15:01:34.004442 31512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "59af68efef5b4b44afa2de222e61d1d8"
format_stamp: "Formatted at 2026-06-19 15:01:33 on dist-test-slave-76rk"
server_key: "6da14c5ff5e582a47038e682d21e4ba4"
server_key_iv: "f740766f3a5166d85c9138f80c21f4e7"
server_key_version: "encryptionkey@0"
I20260619 15:01:34.004691 31512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:34.025134 31512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:34.029996 31512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:34.030349 31512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:34.031441 31512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:34.032868 31512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:34.032977 31512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:34.033142 31512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:34.033216 31512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:34.068825 31512 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:45869
I20260619 15:01:34.068894 31639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:45869 every 8 connection(s)
I20260619 15:01:34.070293 31512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 15:01:34.076402 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 31512
I20260619 15:01:34.076637 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 15:01:34.077231 11967 external_mini_cluster.cc:1428] Setting key 478b6675dfcfa88e5a12cca8f834618e
Jun 19 15:01:34 dist-test-slave-76rk krb5kdc[31272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881293, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.194@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:34.094944 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.073630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:35793 (local address 127.11.175.254:35559)
0619 15:01:34.074104 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:34.074109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:34.075108 (+ 999us) server_negotiation.cc:407] Connection header received
0619 15:01:34.077638 (+ 2530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:34.077643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:34.077717 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:34.077840 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:34.080091 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.080846 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.083522 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.083977 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.084076 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.088603 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:34.088609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:34.088612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:34.088648 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:34.091507 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:34.092342 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:34.092355 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:34.092359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:34.092494 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:34.092980 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:34.092985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:34.092987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:34.093287 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:34.093458 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:34.094251 (+ 793us) server_negotiation.cc:299] Negotiation successful
0619 15:01:34.094514 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":252,"threads_started":1}
I20260619 15:01:34.096118 31643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.074120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35559 (local address 127.11.175.194:35793)
0619 15:01:34.074931 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:34.074979 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:34.077288 (+ 2309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:34.078200 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:34.078209 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:34.079043 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:34.079853 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.079882 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.081024 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.081033 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.083359 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.083378 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.084229 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.084238 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.084571 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.085416 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:01:34.085458 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:34.088322 (+ 2864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:34.091693 (+ 3371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:34.091703 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:34.091723 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:34.092197 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:34.092615 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:34.092620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:34.092624 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:34.092841 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:34.093429 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:34.093438 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:34.093840 (+ 402us) client_negotiation.cc:769] Sending connection context
0619 15:01:34.094140 (+ 300us) client_negotiation.cc:240] Negotiation successful
0619 15:01:34.094793 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":278,"threads_started":1}
I20260619 15:01:34.097956 31640 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35559
I20260619 15:01:34.098328 31640 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:34.099335 31640 heartbeater.cc:511] Master 127.11.175.254:35559 requested a full tablet report, sending...
I20260619 15:01:34.101214 31317 ts_manager.cc:194] Registered new tserver with Master: 59af68efef5b4b44afa2de222e61d1d8 (127.11.175.194:45869)
I20260619 15:01:34.102571 31317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.194@KRBTEST.COM'} at 127.11.175.194:35793
WARNING: no policy specified for kudu/127.11.175.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
Entry for principal kudu/127.11.175.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab.
I20260619 15:01:34.142376 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35559
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:34.500255 31646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:34.500664 31646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:34.500854 31646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:34.509835 31646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:34.509958 31646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:34.510082 31646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 15:01:34.520648 31646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.195.keytab
--principal=kudu/127.11.175.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:35559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:34.522012 31646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:34.523554 31646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:34.537740 31651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:34.537631 31654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:34.537779 31652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:34.539192 31646 server_base.cc:1034] running on GCE node
I20260619 15:01:34.540030 31646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:34.541417 31646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:34.542754 31646 hybrid_clock.cc:648] HybridClock initialized: now 1781881294542717 us; error 51 us; skew 500 ppm
I20260619 15:01:34.543181 31646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:34 dist-test-slave-76rk krb5kdc[31272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881294, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:34.546622 31646 init.cc:377] Logged in from keytab as kudu/127.11.175.195@KRBTEST.COM (short username kudu)
I20260619 15:01:34.548331 31646 webserver.cc:460] Webserver started at http://127.11.175.195:38067/ using document root <none> and password file <none>
I20260619 15:01:34.549018 31646 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:34.549124 31646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:34.549399 31646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:34.552542 31646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43"
format_stamp: "Formatted at 2026-06-19 15:01:34 on dist-test-slave-76rk"
server_key: "97d4d01318053d807484d7ff23d7d87c"
server_key_iv: "5df6c7f2054cf1d3c1883b6faf61258d"
server_key_version: "encryptionkey@0"
I20260619 15:01:34.553320 31646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43"
format_stamp: "Formatted at 2026-06-19 15:01:34 on dist-test-slave-76rk"
server_key: "97d4d01318053d807484d7ff23d7d87c"
server_key_iv: "5df6c7f2054cf1d3c1883b6faf61258d"
server_key_version: "encryptionkey@0"
I20260619 15:01:34.557139 31506 heartbeater.cc:503] Master 127.11.175.254:35559 was elected leader, sending a full tablet report...
I20260619 15:01:34.559603 31646 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260619 15:01:34.563555 31661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:34.565587 31646 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 15:01:34.565810 31646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43"
format_stamp: "Formatted at 2026-06-19 15:01:34 on dist-test-slave-76rk"
server_key: "97d4d01318053d807484d7ff23d7d87c"
server_key_iv: "5df6c7f2054cf1d3c1883b6faf61258d"
server_key_version: "encryptionkey@0"
I20260619 15:01:34.566040 31646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:34.585696 31646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:34.591156 31646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:34.591511 31646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:34.592662 31646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:34.594183 31646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:34.594281 31646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:34.594436 31646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:34.594493 31646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:34.631273 31646 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:39329
I20260619 15:01:34.631435 31773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:39329 every 8 connection(s)
I20260619 15:01:34.632704 31646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 15:01:34.642015 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 31646
I20260619 15:01:34.642208 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 15:01:34.642853 11967 external_mini_cluster.cc:1428] Setting key bdfefa39322f17aa5eaefdd509fdf256
Jun 19 15:01:34 dist-test-slave-76rk krb5kdc[31272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881294, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.195@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:01:34.656519 31776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.635830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:45771 (local address 127.11.175.254:35559)
0619 15:01:34.636335 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:34.636339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:34.637262 (+ 923us) server_negotiation.cc:407] Connection header received
0619 15:01:34.639231 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:34.639237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:34.639348 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:34.639475 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:34.641114 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.642348 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.644519 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.645364 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.645528 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.649517 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:34.649532 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:34.649535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:34.649576 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:34.652640 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:34.653689 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:34.653694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:34.653696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:34.653864 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:34.654401 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:34.654406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:34.654408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:34.654826 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:34.654968 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:34.655919 (+ 951us) server_negotiation.cc:299] Negotiation successful
0619 15:01:34.656182 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":265,"threads_started":1}
I20260619 15:01:34.657824 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.636291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:35559 (local address 127.11.175.195:45771)
0619 15:01:34.637032 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:34.637078 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:34.638920 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:34.639743 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:34.639752 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:34.640317 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:34.640920 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.640937 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.642570 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.642577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.644350 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.644365 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.645634 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.645645 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.645957 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.646801 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:01:34.646833 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:34.649213 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:34.652891 (+ 3678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:34.652900 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:34.652918 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:34.653488 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:34.654016 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:01:34.654020 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:01:34.654023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:01:34.654256 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:01:34.654993 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:34.655001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:34.655421 (+ 420us) client_negotiation.cc:769] Sending connection context
0619 15:01:34.655782 (+ 361us) client_negotiation.cc:240] Negotiation successful
0619 15:01:34.656285 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":273,"threads_started":1}
I20260619 15:01:34.659854 31774 heartbeater.cc:347] Connected to a master server at 127.11.175.254:35559
I20260619 15:01:34.660178 31774 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:34.660933 31774 heartbeater.cc:511] Master 127.11.175.254:35559 requested a full tablet report, sending...
I20260619 15:01:34.662549 31317 ts_manager.cc:194] Registered new tserver with Master: f4c1cdfeb7d24c3db6c92929d9a58f43 (127.11.175.195:39329)
I20260619 15:01:34.663453 31317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.175.195@KRBTEST.COM'} at 127.11.175.195:45771
I20260619 15:01:34.672772 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 15:01:34.691874 31776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.677313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.11.175.254:35559)
0619 15:01:34.677531 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:34.677540 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:34.677981 (+ 441us) server_negotiation.cc:407] Connection header received
0619 15:01:34.678183 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:34.678189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:34.678284 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:34.678395 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:34.679961 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.680703 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.682035 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.682495 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.682641 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.685460 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:34.685466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:34.685470 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:34.685513 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:34.688837 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:34.689640 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:34.689647 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:34.689650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:34.689804 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:34.690201 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:34.690209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:34.690215 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:34.690583 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:34.690895 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:34.691283 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 15:01:34.691542 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 15:01:34.697013 31317 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 15:01:34.704619 31317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 15:01:34.731045 31791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.721563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:45869 (local address 127.0.0.1:42882)
0619 15:01:34.722193 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:34.722210 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:34.722326 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:34.723510 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:34.723515 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:34.723542 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:34.723943 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.723955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.725853 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.725860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.728403 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.728419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.729916 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.729921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.730141 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.730207 (+ 66us) client_negotiation.cc:769] Sending connection context
0619 15:01:34.730348 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 15:01:34.730573 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":149,"thread_start_us":740,"threads_started":1}
I20260619 15:01:34.730959 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.722112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.11.175.194:45869)
0619 15:01:34.722840 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:34.722849 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:34.722880 (+ 31us) server_negotiation.cc:407] Connection header received
0619 15:01:34.722986 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:34.722996 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:34.723287 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:34.724170 (+ 883us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:34.724381 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.725689 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.728619 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.729656 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.730106 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.730313 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:34.730388 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 15:01:34.730560 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":428,"threads_started":1}
I20260619 15:01:34.731854 31789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.720923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:39329 (local address 127.0.0.1:36554)
0619 15:01:34.721512 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:34.721531 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:34.721655 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:34.724098 (+ 2443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:34.724103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:34.724131 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:34.724492 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.724502 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.726448 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.726453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.728487 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.728535 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.730488 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.730496 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.730715 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.730753 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 15:01:34.731245 (+ 492us) client_negotiation.cc:240] Negotiation successful
0619 15:01:34.731400 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":6,"spinlock_wait_cycles":3456,"thread_start_us":314,"threads_started":1}
I20260619 15:01:34.732712 31790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.721255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36554 (local address 127.11.175.195:39329)
0619 15:01:34.723320 (+ 2065us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:34.723330 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:34.723364 (+ 34us) server_negotiation.cc:407] Connection header received
0619 15:01:34.723455 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:34.723464 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:34.723715 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:34.724123 (+ 408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:34.724667 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.726278 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.728671 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.730255 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.730486 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.730677 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:34.732112 (+ 1435us) server_negotiation.cc:299] Negotiation successful
0619 15:01:34.732335 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1870,"thread_start_us":2259,"threads_started":1}
I20260619 15:01:34.732797 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.719524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:44585 (local address 127.0.0.1:41118)
0619 15:01:34.720325 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:34.720371 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:34.720564 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:34.721557 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:34.721570 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:34.721614 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:34.722129 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.722807 (+ 678us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.724896 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.724901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.727456 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.727472 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.729832 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.729840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.732126 (+ 2286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.732161 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 15:01:34.732253 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 15:01:34.732417 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":29,"thread_start_us":352,"threads_started":1}
I20260619 15:01:34.732944 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.719890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41118 (local address 127.11.175.193:44585)
0619 15:01:34.720648 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:34.720669 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:34.720699 (+ 30us) server_negotiation.cc:407] Connection header received
0619 15:01:34.720801 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:34.720807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:34.721033 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:34.721255 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:34.723029 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.724750 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.727656 (+ 2906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.729592 (+ 1936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.729827 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.730026 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:34.732357 (+ 2331us) server_negotiation.cc:299] Negotiation successful
0619 15:01:34.732569 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":417,"threads_started":1}
I20260619 15:01:34.737322 31709 tablet_service.cc:1467] Processing CreateTablet for tablet 1f2d3e65a2ed4f33bc88258a30a2162b (DEFAULT_TABLE table=test-table [id=5d795c10902340e695706d4480ac0fb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:34.737466 31575 tablet_service.cc:1467] Processing CreateTablet for tablet 1f2d3e65a2ed4f33bc88258a30a2162b (DEFAULT_TABLE table=test-table [id=5d795c10902340e695706d4480ac0fb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:34.737350 31441 tablet_service.cc:1467] Processing CreateTablet for tablet 1f2d3e65a2ed4f33bc88258a30a2162b (DEFAULT_TABLE table=test-table [id=5d795c10902340e695706d4480ac0fb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:34.738900 31709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f2d3e65a2ed4f33bc88258a30a2162b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:34.738926 31441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f2d3e65a2ed4f33bc88258a30a2162b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:34.739794 31575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f2d3e65a2ed4f33bc88258a30a2162b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:34.751575 31793 tablet_bootstrap.cc:492] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: Bootstrap starting.
I20260619 15:01:34.754928 31794 tablet_bootstrap.cc:492] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: Bootstrap starting.
I20260619 15:01:34.756688 31793 tablet_bootstrap.cc:654] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:34.756841 31795 tablet_bootstrap.cc:492] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: Bootstrap starting.
I20260619 15:01:34.758205 31793 log.cc:824] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:34.759491 31794 tablet_bootstrap.cc:654] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:34.760552 31795 tablet_bootstrap.cc:654] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:34.760887 31794 log.cc:824] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:34.761798 31795 log.cc:824] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:34.762178 31793 tablet_bootstrap.cc:492] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: No bootstrap required, opened a new log
I20260619 15:01:34.762626 31793 ts_tablet_manager.cc:1397] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260619 15:01:34.769611 31795 tablet_bootstrap.cc:492] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: No bootstrap required, opened a new log
I20260619 15:01:34.770000 31795 ts_tablet_manager.cc:1397] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260619 15:01:34.770360 31794 tablet_bootstrap.cc:492] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: No bootstrap required, opened a new log
I20260619 15:01:34.770744 31794 ts_tablet_manager.cc:1397] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.006s
I20260619 15:01:34.775018 31793 raft_consensus.cc:348] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 } } peers { permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 } } peers { permanent_uuid: "59af68efef5b4b44afa2de222e61d1d8" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45869 } }
I20260619 15:01:34.775542 31793 raft_consensus.cc:374] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:34.775691 31793 raft_consensus.cc:729] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c1cdfeb7d24c3db6c92929d9a58f43, State: Initialized, Role: FOLLOWER
I20260619 15:01:34.776409 31793 consensus_queue.cc:260] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 } } peers { permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 } } peers { permanent_uuid: "59af68efef5b4b44afa2de222e61d1d8" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45869 } }
I20260619 15:01:34.777647 31774 heartbeater.cc:503] Master 127.11.175.254:35559 was elected leader, sending a full tablet report...
I20260619 15:01:34.778394 31793 ts_tablet_manager.cc:1428] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260619 15:01:34.781463 31795 raft_consensus.cc:348] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 } } peers { permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 } } peers { permanent_uuid: "59af68efef5b4b44afa2de222e61d1d8" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45869 } }
I20260619 15:01:34.781427 31794 raft_consensus.cc:348] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 } } peers { permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 } } peers { permanent_uuid: "59af68efef5b4b44afa2de222e61d1d8" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45869 } }
I20260619 15:01:34.781865 31795 raft_consensus.cc:374] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:34.781950 31794 raft_consensus.cc:374] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:34.781991 31795 raft_consensus.cc:729] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59af68efef5b4b44afa2de222e61d1d8, State: Initialized, Role: FOLLOWER
I20260619 15:01:34.782068 31794 raft_consensus.cc:729] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5ecda40d4644f27b44fea8f27936177, State: Initialized, Role: FOLLOWER
I20260619 15:01:34.782778 31795 consensus_queue.cc:260] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 } } peers { permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 } } peers { permanent_uuid: "59af68efef5b4b44afa2de222e61d1d8" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45869 } }
I20260619 15:01:34.782892 31794 consensus_queue.cc:260] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 } } peers { permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 } } peers { permanent_uuid: "59af68efef5b4b44afa2de222e61d1d8" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45869 } }
I20260619 15:01:34.784011 31640 heartbeater.cc:503] Master 127.11.175.254:35559 was elected leader, sending a full tablet report...
I20260619 15:01:34.784756 31795 ts_tablet_manager.cc:1428] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: Time spent starting tablet: real 0.015s user 0.007s sys 0.005s
I20260619 15:01:34.787284 31794 ts_tablet_manager.cc:1428] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
W20260619 15:01:34.823030 31641 tablet.cc:2369] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 15:01:34.884668 31775 tablet.cc:2369] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:01:34.918656 31800 raft_consensus.cc:482] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 15:01:34.919026 31800 raft_consensus.cc:504] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 } } peers { permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 } } peers { permanent_uuid: "59af68efef5b4b44afa2de222e61d1d8" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45869 } }
I20260619 15:01:34.920866 31800 leader_election.cc:288] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5ecda40d4644f27b44fea8f27936177 (127.11.175.193:44585), f4c1cdfeb7d24c3db6c92929d9a58f43 (127.11.175.195:39329)
I20260619 15:01:34.929759 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.921354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:59039 (local address 127.11.175.193:44585)
0619 15:01:34.921613 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:34.921621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:34.922190 (+ 569us) server_negotiation.cc:407] Connection header received
0619 15:01:34.922443 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:34.922451 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:34.922588 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:34.922734 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:34.923406 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.924365 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.927088 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.928446 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.928636 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.928737 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:34.929362 (+ 625us) server_negotiation.cc:299] Negotiation successful
0619 15:01:34.929504 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 15:01:34.929790 31790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.922329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:51203 (local address 127.11.175.195:39329)
0619 15:01:34.922545 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:34.922553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:34.923062 (+ 509us) server_negotiation.cc:407] Connection header received
0619 15:01:34.923186 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:34.923190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:34.923252 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:34.923377 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:34.924008 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.924922 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.927360 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.928626 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.928790 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.928849 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:34.929344 (+ 495us) server_negotiation.cc:299] Negotiation successful
0619 15:01:34.929490 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 15:01:34.929991 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.922179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:39329 (local address 127.11.175.194:51203)
0619 15:01:34.922917 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:34.922941 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:34.923071 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:34.923503 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:34.923508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:34.923534 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:34.923823 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.923830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.925077 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.925084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.927182 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.927193 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.928868 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.928874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.929123 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.929154 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 15:01:34.929236 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 15:01:34.929463 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":89,"thread_start_us":347,"threads_started":1}
I20260619 15:01:34.930017 31802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.921309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:44585 (local address 127.11.175.194:59039)
0619 15:01:34.922017 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:34.922033 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:34.922250 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:34.922756 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:34.922761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:34.922789 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:34.923238 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.923248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.924515 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.924519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.926845 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:34.926870 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.928766 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.928774 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:34.928985 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.929106 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 15:01:34.929232 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 15:01:34.929463 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":119,"thread_start_us":418,"threads_started":1}
I20260619 15:01:34.931222 31729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f2d3e65a2ed4f33bc88258a30a2162b" candidate_uuid: "59af68efef5b4b44afa2de222e61d1d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" is_pre_election: true
I20260619 15:01:34.931262 31461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f2d3e65a2ed4f33bc88258a30a2162b" candidate_uuid: "59af68efef5b4b44afa2de222e61d1d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5ecda40d4644f27b44fea8f27936177" is_pre_election: true
I20260619 15:01:34.931774 31729 raft_consensus.cc:2454] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59af68efef5b4b44afa2de222e61d1d8 in term 0.
I20260619 15:01:34.931860 31461 raft_consensus.cc:2454] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59af68efef5b4b44afa2de222e61d1d8 in term 0.
I20260619 15:01:34.932569 31530 leader_election.cc:302] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59af68efef5b4b44afa2de222e61d1d8, f4c1cdfeb7d24c3db6c92929d9a58f43; no voters:
I20260619 15:01:34.933053 31800 raft_consensus.cc:2781] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 15:01:34.933265 31800 raft_consensus.cc:482] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 15:01:34.933389 31800 raft_consensus.cc:3037] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:34.935845 31800 raft_consensus.cc:504] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 } } peers { permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 } } peers { permanent_uuid: "59af68efef5b4b44afa2de222e61d1d8" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45869 } }
I20260619 15:01:34.936488 31800 leader_election.cc:288] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [CANDIDATE]: Term 1 election: Requested vote from peers a5ecda40d4644f27b44fea8f27936177 (127.11.175.193:44585), f4c1cdfeb7d24c3db6c92929d9a58f43 (127.11.175.195:39329)
I20260619 15:01:34.937189 31461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f2d3e65a2ed4f33bc88258a30a2162b" candidate_uuid: "59af68efef5b4b44afa2de222e61d1d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5ecda40d4644f27b44fea8f27936177"
I20260619 15:01:34.937331 31729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f2d3e65a2ed4f33bc88258a30a2162b" candidate_uuid: "59af68efef5b4b44afa2de222e61d1d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43"
I20260619 15:01:34.937464 31461 raft_consensus.cc:3037] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:34.937611 31729 raft_consensus.cc:3037] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:34.939918 31461 raft_consensus.cc:2454] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59af68efef5b4b44afa2de222e61d1d8 in term 1.
I20260619 15:01:34.940235 31729 raft_consensus.cc:2454] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59af68efef5b4b44afa2de222e61d1d8 in term 1.
I20260619 15:01:34.940514 31530 leader_election.cc:302] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59af68efef5b4b44afa2de222e61d1d8, a5ecda40d4644f27b44fea8f27936177; no voters:
I20260619 15:01:34.940919 31800 raft_consensus.cc:2781] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:34.941512 31800 raft_consensus.cc:686] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 1 LEADER]: Becoming Leader. State: Replica: 59af68efef5b4b44afa2de222e61d1d8, State: Running, Role: LEADER
I20260619 15:01:34.942130 31800 consensus_queue.cc:237] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 } } peers { permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 } } peers { permanent_uuid: "59af68efef5b4b44afa2de222e61d1d8" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45869 } }
I20260619 15:01:34.947193 31316 catalog_manager.cc:5495] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59af68efef5b4b44afa2de222e61d1d8 (127.11.175.194). New cstate: current_term: 1 leader_uuid: "59af68efef5b4b44afa2de222e61d1d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59af68efef5b4b44afa2de222e61d1d8" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 45869 } health_report { overall_health: HEALTHY } } }
I20260619 15:01:34.974752 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:34.969003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42888 (local address 127.11.175.194:45869)
0619 15:01:34.969248 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:34.969253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:34.969274 (+ 21us) server_negotiation.cc:407] Connection header received
0619 15:01:34.969552 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:34.969557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:34.969642 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:34.969859 (+ 217us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:01:34.970225 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.971056 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.972457 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:34.972867 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:34.973002 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:34.973538 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:01:34.974070 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:01:34.974256 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:34.974309 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 15:01:34.974396 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260619 15:01:34.984100 31729 raft_consensus.cc:1261] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43 [term 1 FOLLOWER]: Refusing update from remote peer 59af68efef5b4b44afa2de222e61d1d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:34.984191 31461 raft_consensus.cc:1261] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177 [term 1 FOLLOWER]: Refusing update from remote peer 59af68efef5b4b44afa2de222e61d1d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:34.985319 31804 consensus_queue.cc:1035] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c1cdfeb7d24c3db6c92929d9a58f43" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 39329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:34.985709 31800 consensus_queue.cc:1035] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5ecda40d4644f27b44fea8f27936177" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 44585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:35.013721 31317 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:35.014075 31317 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:35.018533 31317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e51c0394ca6c4afeb2a179897e4f3380: Sending DeleteTablet for 3 replicas of tablet 1f2d3e65a2ed4f33bc88258a30a2162b
I20260619 15:01:35.019917 31441 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2d3e65a2ed4f33bc88258a30a2162b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:35 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:41118
I20260619 15:01:35.020048 31575 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2d3e65a2ed4f33bc88258a30a2162b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:35 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:42882
I20260619 15:01:35.020048 31709 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2d3e65a2ed4f33bc88258a30a2162b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:35 UTC) from {username='kudu', principal='kudu/127.11.175.254@KRBTEST.COM'} at 127.0.0.1:36554
I20260619 15:01:35.023481 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 31378
I20260619 15:01:35.031320 31818 tablet_replica.cc:330] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: stopping tablet replica
I20260619 15:01:35.031553 31819 tablet_replica.cc:330] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: stopping tablet replica
I20260619 15:01:35.031875 31818 raft_consensus.cc:2229] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:35.032056 31819 raft_consensus.cc:2229] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 15:01:35.032627 31818 raft_consensus.cc:2258] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:35.032814 31819 raft_consensus.cc:2258] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:35.033623 31817 tablet_replica.cc:330] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: stopping tablet replica
I20260619 15:01:35.034003 31817 raft_consensus.cc:2229] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:35.034530 31817 raft_consensus.cc:2258] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:35.035454 31819 ts_tablet_manager.cc:1905] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:35.038172 31817 ts_tablet_manager.cc:1905] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:35.043671 31819 ts_tablet_manager.cc:1918] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:35.043991 31819 log.cc:1196] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/1f2d3e65a2ed4f33bc88258a30a2162b
Jun 19 15:01:35 dist-test-slave-76rk krb5kdc[31272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.193@KRBTEST.COM
I20260619 15:01:35.044708 31819 ts_tablet_manager.cc:1939] T 1f2d3e65a2ed4f33bc88258a30a2162b P 59af68efef5b4b44afa2de222e61d1d8: Deleting consensus metadata
I20260619 15:01:35.045878 31817 ts_tablet_manager.cc:1918] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:35.046067 31817 log.cc:1196] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-0/wal/wals/1f2d3e65a2ed4f33bc88258a30a2162b
I20260619 15:01:35.046715 31817 ts_tablet_manager.cc:1939] T 1f2d3e65a2ed4f33bc88258a30a2162b P a5ecda40d4644f27b44fea8f27936177: Deleting consensus metadata
I20260619 15:01:35.047659 31303 catalog_manager.cc:4841] TS 59af68efef5b4b44afa2de222e61d1d8 (127.11.175.194:45869): tablet 1f2d3e65a2ed4f33bc88258a30a2162b (table test-table [id=5d795c10902340e695706d4480ac0fb5]) successfully deleted
I20260619 15:01:35.048929 31303 catalog_manager.cc:4841] TS a5ecda40d4644f27b44fea8f27936177 (127.11.175.193:44585): tablet 1f2d3e65a2ed4f33bc88258a30a2162b (table test-table [id=5d795c10902340e695706d4480ac0fb5]) successfully deleted
I20260619 15:01:35.049078 31818 ts_tablet_manager.cc:1905] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:35.054492 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:35.027170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.11.175.193:44585)
0619 15:01:35.027746 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:35.027755 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:35.034185 (+ 6430us) server_negotiation.cc:407] Connection header received
0619 15:01:35.034297 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:35.034302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:35.034400 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:35.034522 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:35.036268 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:35.037417 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:35.039332 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:35.040023 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:35.040129 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:35.046360 (+ 6231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:35.046379 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:35.046401 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:35.046458 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:35.051204 (+ 4746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:35.052092 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:35.052100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:35.052102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:35.052254 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:35.052690 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:35.052700 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:35.052702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:35.053078 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:35.053400 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:35.053760 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 15:01:35.053988 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 15:01:35.055328 31501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:35.057178 31818 ts_tablet_manager.cc:1918] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:35.057514 31818 log.cc:1196] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/1f2d3e65a2ed4f33bc88258a30a2162b
I20260619 15:01:35.058502 31818 ts_tablet_manager.cc:1939] T 1f2d3e65a2ed4f33bc88258a30a2162b P f4c1cdfeb7d24c3db6c92929d9a58f43: Deleting consensus metadata
I20260619 15:01:35.060814 31303 catalog_manager.cc:4841] TS f4c1cdfeb7d24c3db6c92929d9a58f43 (127.11.175.195:39329): tablet 1f2d3e65a2ed4f33bc88258a30a2162b (table test-table [id=5d795c10902340e695706d4480ac0fb5]) successfully deleted
I20260619 15:01:35.149242 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 31378
I20260619 15:01:35.168135 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 31512
Jun 19 15:01:35 dist-test-slave-76rk krb5kdc[31272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.194@KRBTEST.COM
I20260619 15:01:35.186276 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:35.169263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42902 (local address 127.11.175.194:45869)
0619 15:01:35.169446 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:35.169451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:35.169466 (+ 15us) server_negotiation.cc:407] Connection header received
0619 15:01:35.169527 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:35.169530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:35.169589 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:35.169761 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:35.171420 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:35.172283 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:35.173593 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:35.174106 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:35.174259 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:35.179392 (+ 5133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:35.179411 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:35.179426 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:35.179474 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:35.182831 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:35.183892 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:35.183901 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:35.183903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:35.184075 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:35.184546 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:35.184551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:35.184553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:35.184988 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:35.185320 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:35.185704 (+ 384us) server_negotiation.cc:299] Negotiation successful
0619 15:01:35.185896 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 15:01:35.187077 31635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:35.303983 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 31512
I20260619 15:01:35.323680 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 31646
Jun 19 15:01:35 dist-test-slave-76rk krb5kdc[31272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.195@KRBTEST.COM
I20260619 15:01:35.340562 31790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:35.324541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36558 (local address 127.11.175.195:39329)
0619 15:01:35.324760 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:35.324765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:35.324786 (+ 21us) server_negotiation.cc:407] Connection header received
0619 15:01:35.324845 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:35.324849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:35.324913 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:35.325005 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:35.326389 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:35.327285 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:35.328739 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:35.329169 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:35.329282 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:35.334181 (+ 4899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:35.334230 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:35.334246 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:35.334296 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:35.337368 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:35.338227 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:35.338235 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:35.338238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:35.338443 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:35.339003 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:35.339008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:35.339010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:35.339433 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:35.339679 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:35.340052 (+ 373us) server_negotiation.cc:299] Negotiation successful
0619 15:01:35.340292 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260619 15:01:35.341388 31769 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:35.435117 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 31646
I20260619 15:01:35.452759 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 31285
I20260619 15:01:35.454047 31347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:35.569919 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 31285
2026-06-19T15:01:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4654 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T15:01:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:35Z Disabled control of system clock
I20260619 15:01:35.655299 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:36955
--webserver_interface=127.11.175.254
--webserver_port=0
--builtin_ntp_servers=127.11.175.212:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:36955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 15:01:36.007588 31828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:36.007928 31828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:36.008015 31828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:36.015724 31828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:36.015859 31828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:36.015899 31828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:36.015951 31828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:36.015997 31828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:36.026468 31828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:36955
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:36955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:36.028097 31828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:36.030239 31828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:36.042915 31833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:36.043051 31834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:36.043023 31836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:36.044224 31828 server_base.cc:1034] running on GCE node
I20260619 15:01:36.045212 31828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:36.046900 31828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:36.048240 31828 hybrid_clock.cc:648] HybridClock initialized: now 1781881296048158 us; error 57 us; skew 500 ppm
I20260619 15:01:36.048659 31828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:36.051600 31828 webserver.cc:460] Webserver started at http://127.11.175.254:33627/ using document root <none> and password file <none>
I20260619 15:01:36.052390 31828 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:36.052531 31828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:36.052870 31828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:36.056684 31828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "035169628b8944d9b3d37156bc0f8e9d"
format_stamp: "Formatted at 2026-06-19 15:01:36 on dist-test-slave-76rk"
server_key: "52eb0505e0371ba1c1b6b925097510e2"
server_key_iv: "36502347dc3220986827adb7437910eb"
server_key_version: "encryptionkey@0"
I20260619 15:01:36.057693 31828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "035169628b8944d9b3d37156bc0f8e9d"
format_stamp: "Formatted at 2026-06-19 15:01:36 on dist-test-slave-76rk"
server_key: "52eb0505e0371ba1c1b6b925097510e2"
server_key_iv: "36502347dc3220986827adb7437910eb"
server_key_version: "encryptionkey@0"
I20260619 15:01:36.063560 31828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 15:01:36.067495 31842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:36.069093 31828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 15:01:36.069414 31828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "035169628b8944d9b3d37156bc0f8e9d"
format_stamp: "Formatted at 2026-06-19 15:01:36 on dist-test-slave-76rk"
server_key: "52eb0505e0371ba1c1b6b925097510e2"
server_key_iv: "36502347dc3220986827adb7437910eb"
server_key_version: "encryptionkey@0"
I20260619 15:01:36.069823 31828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:36.093204 31828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:36.094177 31828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:36.094519 31828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:36.114590 31828 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:36955
I20260619 15:01:36.114614 31893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:36955 every 8 connection(s)
I20260619 15:01:36.116008 31828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:36.120000 31894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:36.123934 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 31828
I20260619 15:01:36.124178 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:36.124809 11967 external_mini_cluster.cc:1428] Setting key 78c12f2fca1d318beb9c930f235f3ac8
I20260619 15:01:36.133663 31894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d: Bootstrap starting.
I20260619 15:01:36.137617 31894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:36.139219 31894 log.cc:824] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:36.143082 31894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d: No bootstrap required, opened a new log
I20260619 15:01:36.143316 31897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:36.127106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59634 (local address 127.11.175.254:36955)
0619 15:01:36.128280 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:36.128297 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:36.128353 (+ 56us) server_negotiation.cc:407] Connection header received
0619 15:01:36.131622 (+ 3269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:36.131658 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:36.132347 (+ 689us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:36.132826 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:36.133430 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:36.135485 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:36.137107 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:36.138317 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:36.138522 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:36.140158 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:36.140169 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:36.140185 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:36.140876 (+ 691us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:36.141000 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:36.141029 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:36.141162 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:36.141503 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 15:01:36.141874 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":366,"threads_started":1}
I20260619 15:01:36.151500 31894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "035169628b8944d9b3d37156bc0f8e9d" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36955 } }
I20260619 15:01:36.151913 31894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:36.151990 31894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 035169628b8944d9b3d37156bc0f8e9d, State: Initialized, Role: FOLLOWER
I20260619 15:01:36.152581 31894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "035169628b8944d9b3d37156bc0f8e9d" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36955 } }
I20260619 15:01:36.152854 31894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:36.153026 31894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:36.153225 31894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:36.155773 31894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "035169628b8944d9b3d37156bc0f8e9d" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36955 } }
I20260619 15:01:36.156309 31894 leader_election.cc:302] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 035169628b8944d9b3d37156bc0f8e9d; no voters:
I20260619 15:01:36.156926 31894 leader_election.cc:288] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:36.157202 31899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:36.157945 31899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [term 1 LEADER]: Becoming Leader. State: Replica: 035169628b8944d9b3d37156bc0f8e9d, State: Running, Role: LEADER
I20260619 15:01:36.158519 31899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "035169628b8944d9b3d37156bc0f8e9d" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36955 } }
I20260619 15:01:36.159433 31894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:36.161815 31901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 035169628b8944d9b3d37156bc0f8e9d. Latest consensus state: current_term: 1 leader_uuid: "035169628b8944d9b3d37156bc0f8e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "035169628b8944d9b3d37156bc0f8e9d" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36955 } } }
I20260619 15:01:36.161756 31900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "035169628b8944d9b3d37156bc0f8e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "035169628b8944d9b3d37156bc0f8e9d" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36955 } } }
I20260619 15:01:36.162344 31900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:36.163051 31901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:36.163195 31906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:36.168499 31906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:36.178864 31906 catalog_manager.cc:1347] Generated new cluster ID: 440b3335b8764e478cd1d2238eacd5c7
I20260619 15:01:36.179004 31906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:36.196514 31906 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:36.198500 31906 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:36.207705 31906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d: Generated new TSK 0
I20260619 15:01:36.208855 31906 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:36.218305 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.193:0
--local_ip_for_outbound_sockets=127.11.175.193
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36955
--builtin_ntp_servers=127.11.175.212:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 15:01:36.570083 31918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:36.570452 31918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:36.570604 31918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:36.578389 31918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:36.578543 31918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:36.578758 31918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.193
I20260619 15:01:36.588353 31918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.175.193
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.193
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:36.589706 31918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:36.591663 31918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:36.606365 31926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:36.606369 31924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:36.607414 31918 server_base.cc:1034] running on GCE node
W20260619 15:01:36.606415 31923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:36.608566 31918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:36.609637 31918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:36.610911 31918 hybrid_clock.cc:648] HybridClock initialized: now 1781881296610855 us; error 46 us; skew 500 ppm
I20260619 15:01:36.611336 31918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:36.614041 31918 webserver.cc:460] Webserver started at http://127.11.175.193:42517/ using document root <none> and password file <none>
I20260619 15:01:36.614908 31918 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:36.615056 31918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:36.615397 31918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:36.618559 31918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/data/instance:
uuid: "58460e7dc7224cbd9eec338c6a2aa62d"
format_stamp: "Formatted at 2026-06-19 15:01:36 on dist-test-slave-76rk"
server_key: "7368e76c4e6dee11dcdc01c8ff557d72"
server_key_iv: "48abcb6739aa23b0caecd725ef1268c1"
server_key_version: "encryptionkey@0"
I20260619 15:01:36.619531 31918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance:
uuid: "58460e7dc7224cbd9eec338c6a2aa62d"
format_stamp: "Formatted at 2026-06-19 15:01:36 on dist-test-slave-76rk"
server_key: "7368e76c4e6dee11dcdc01c8ff557d72"
server_key_iv: "48abcb6739aa23b0caecd725ef1268c1"
server_key_version: "encryptionkey@0"
I20260619 15:01:36.625658 31918 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260619 15:01:36.629793 31932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:36.631465 31918 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260619 15:01:36.631681 31918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/wal
uuid: "58460e7dc7224cbd9eec338c6a2aa62d"
format_stamp: "Formatted at 2026-06-19 15:01:36 on dist-test-slave-76rk"
server_key: "7368e76c4e6dee11dcdc01c8ff557d72"
server_key_iv: "48abcb6739aa23b0caecd725ef1268c1"
server_key_version: "encryptionkey@0"
I20260619 15:01:36.631933 31918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:36.648043 31918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:36.648945 31918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:36.649291 31918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:36.650370 31918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:36.652019 31918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:36.652148 31918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:36.652315 31918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:36.652405 31918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:36.688405 31918 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.193:38299
I20260619 15:01:36.688428 32044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.193:38299 every 8 connection(s)
I20260619 15:01:36.689759 31918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/data/info.pb
I20260619 15:01:36.695964 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 31918
I20260619 15:01:36.696164 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-0/wal/instance
I20260619 15:01:36.696712 11967 external_mini_cluster.cc:1428] Setting key 5942cd466447c43bf6f62be2d57f5758
I20260619 15:01:36.700115 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.194:0
--local_ip_for_outbound_sockets=127.11.175.194
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36955
--builtin_ntp_servers=127.11.175.212:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 15:01:36.704909 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:36.692486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:38975 (local address 127.11.175.254:36955)
0619 15:01:36.692966 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:36.692973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:36.694226 (+ 1253us) server_negotiation.cc:407] Connection header received
0619 15:01:36.696501 (+ 2275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:36.696505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:36.696592 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:36.696711 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:36.698033 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:36.699125 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:36.701774 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:36.702170 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:36.702282 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:36.704038 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:36.704043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:36.704046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:36.704065 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:36.704129 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:36.704141 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:36.704218 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:36.704537 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 15:01:36.704637 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":228,"threads_started":1}
I20260619 15:01:36.706475 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:36.693142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:36955 (local address 127.11.175.193:38975)
0619 15:01:36.694051 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:36.694089 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:36.696118 (+ 2029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:36.697054 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:36.697067 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:36.697201 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:36.697865 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:36.697882 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:36.699307 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:36.699313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:36.701604 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:36.701622 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:36.702421 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:36.702430 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:36.702811 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:36.703642 (+ 831us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 15:01:36.703648 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:36.703715 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 15:01:36.703830 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:36.704248 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:36.704270 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:36.704281 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 15:01:36.704479 (+ 198us) client_negotiation.cc:240] Negotiation successful
0619 15:01:36.704950 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":274,"threads_started":1}
I20260619 15:01:36.708392 32045 heartbeater.cc:347] Connected to a master server at 127.11.175.254:36955
I20260619 15:01:36.708739 32045 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:36.709725 32045 heartbeater.cc:511] Master 127.11.175.254:36955 requested a full tablet report, sending...
I20260619 15:01:36.712411 31859 ts_manager.cc:194] Registered new tserver with Master: 58460e7dc7224cbd9eec338c6a2aa62d (127.11.175.193:38299)
I20260619 15:01:36.714663 31859 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.193:38975
W20260619 15:01:37.060720 32049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:37.061105 32049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:37.061203 32049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:37.068874 32049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:37.068995 32049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:37.069109 32049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.194
I20260619 15:01:37.079157 32049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.175.194
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.194
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:37.080464 32049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:37.082442 32049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:37.098356 32054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:37.098387 32055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:37.098454 32057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:37.100060 32049 server_base.cc:1034] running on GCE node
I20260619 15:01:37.100800 32049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:37.102126 32049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:37.103753 32049 hybrid_clock.cc:648] HybridClock initialized: now 1781881297103669 us; error 76 us; skew 500 ppm
I20260619 15:01:37.104151 32049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:37.107043 32049 webserver.cc:460] Webserver started at http://127.11.175.194:34393/ using document root <none> and password file <none>
I20260619 15:01:37.107844 32049 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:37.107988 32049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:37.108485 32049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:37.111611 32049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/data/instance:
uuid: "0307672fbd3c4ee3b0536f9e90306a77"
format_stamp: "Formatted at 2026-06-19 15:01:37 on dist-test-slave-76rk"
server_key: "7101f37112e4ae8ade0d333ea07013d9"
server_key_iv: "e5563bf6112784040ef28b61134053e9"
server_key_version: "encryptionkey@0"
I20260619 15:01:37.112509 32049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance:
uuid: "0307672fbd3c4ee3b0536f9e90306a77"
format_stamp: "Formatted at 2026-06-19 15:01:37 on dist-test-slave-76rk"
server_key: "7101f37112e4ae8ade0d333ea07013d9"
server_key_iv: "e5563bf6112784040ef28b61134053e9"
server_key_version: "encryptionkey@0"
I20260619 15:01:37.120399 32049 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260619 15:01:37.125444 32063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:37.127385 32049 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:01:37.127594 32049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/wal
uuid: "0307672fbd3c4ee3b0536f9e90306a77"
format_stamp: "Formatted at 2026-06-19 15:01:37 on dist-test-slave-76rk"
server_key: "7101f37112e4ae8ade0d333ea07013d9"
server_key_iv: "e5563bf6112784040ef28b61134053e9"
server_key_version: "encryptionkey@0"
I20260619 15:01:37.127871 32049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:37.146121 32049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:37.147082 32049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:37.147357 32049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:37.148341 32049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:37.149904 32049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:37.149991 32049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:37.150070 32049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:37.150127 32049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:37.186079 32049 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.194:37991
I20260619 15:01:37.186172 32175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.194:37991 every 8 connection(s)
I20260619 15:01:37.187517 32049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/data/info.pb
I20260619 15:01:37.193430 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 32049
I20260619 15:01:37.193608 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/wal/instance
I20260619 15:01:37.194258 11967 external_mini_cluster.cc:1428] Setting key 5b2bd95b38ce84a0f42719148a5a39f3
I20260619 15:01:37.197633 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.175.195:0
--local_ip_for_outbound_sockets=127.11.175.195
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36955
--builtin_ntp_servers=127.11.175.212:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 15:01:37.201697 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.190399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.194:41673 (local address 127.11.175.254:36955)
0619 15:01:37.190580 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:37.190584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:37.192342 (+ 1758us) server_negotiation.cc:407] Connection header received
0619 15:01:37.194378 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:37.194384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:37.194488 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:37.194607 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:37.195983 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.196865 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.198724 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.199142 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.199294 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.200835 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:37.200840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:37.200843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:37.200865 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:37.200899 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:37.200911 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:37.201008 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:37.201263 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 15:01:37.201378 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 15:01:37.202987 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.190934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:36955 (local address 127.11.175.194:41673)
0619 15:01:37.192137 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:37.192182 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:37.194039 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:37.194914 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:37.194924 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:37.195027 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:37.195802 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.195821 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.197043 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.197047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:37.198521 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.198537 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.199396 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.199404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:37.199652 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.200457 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 15:01:37.200460 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:37.200506 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 15:01:37.200596 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:37.201022 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:37.201038 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:37.201051 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 15:01:37.201253 (+ 202us) client_negotiation.cc:240] Negotiation successful
0619 15:01:37.201723 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":537,"threads_started":1}
I20260619 15:01:37.204785 32176 heartbeater.cc:347] Connected to a master server at 127.11.175.254:36955
I20260619 15:01:37.205132 32176 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:37.205973 32176 heartbeater.cc:511] Master 127.11.175.254:36955 requested a full tablet report, sending...
I20260619 15:01:37.207846 31859 ts_manager.cc:194] Registered new tserver with Master: 0307672fbd3c4ee3b0536f9e90306a77 (127.11.175.194:37991)
I20260619 15:01:37.208743 31859 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.194:41673
W20260619 15:01:37.545053 32179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:37.545349 32179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:37.545499 32179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:37.552996 32179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:37.553118 32179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:37.553303 32179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.175.195
I20260619 15:01:37.562968 32179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.175.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.175.195
--webserver_port=0
--tserver_master_addrs=127.11.175.254:36955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.175.195
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:37.564318 32179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:37.566220 32179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:37.580845 32184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:37.580904 32187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:37.580788 32185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:37.582341 32179 server_base.cc:1034] running on GCE node
I20260619 15:01:37.583213 32179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:37.584308 32179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:37.585578 32179 hybrid_clock.cc:648] HybridClock initialized: now 1781881297585538 us; error 33 us; skew 500 ppm
I20260619 15:01:37.585976 32179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 15:01:37.588909 32179 webserver.cc:460] Webserver started at http://127.11.175.195:38605/ using document root <none> and password file <none>
I20260619 15:01:37.589684 32179 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:37.589833 32179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:37.590179 32179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:37.593395 32179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/data/instance:
uuid: "d7cf49afafb644ac9c26751d0c0ab773"
format_stamp: "Formatted at 2026-06-19 15:01:37 on dist-test-slave-76rk"
server_key: "3ea577c5bba56b2b12773e90fc6030fe"
server_key_iv: "ff25e6deeeb85ba9674ce09d65bbb0cb"
server_key_version: "encryptionkey@0"
I20260619 15:01:37.594303 32179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance:
uuid: "d7cf49afafb644ac9c26751d0c0ab773"
format_stamp: "Formatted at 2026-06-19 15:01:37 on dist-test-slave-76rk"
server_key: "3ea577c5bba56b2b12773e90fc6030fe"
server_key_iv: "ff25e6deeeb85ba9674ce09d65bbb0cb"
server_key_version: "encryptionkey@0"
I20260619 15:01:37.600533 32179 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260619 15:01:37.604384 32193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:37.606060 32179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 15:01:37.606293 32179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/wal
uuid: "d7cf49afafb644ac9c26751d0c0ab773"
format_stamp: "Formatted at 2026-06-19 15:01:37 on dist-test-slave-76rk"
server_key: "3ea577c5bba56b2b12773e90fc6030fe"
server_key_iv: "ff25e6deeeb85ba9674ce09d65bbb0cb"
server_key_version: "encryptionkey@0"
I20260619 15:01:37.606546 32179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:37.618749 32179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:37.619642 32179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:37.619967 32179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:37.621027 32179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 15:01:37.622627 32179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 15:01:37.622856 32179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 15:01:37.623003 32179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 15:01:37.623092 32179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:37.661900 32179 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.195:33331
I20260619 15:01:37.662000 32305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.195:33331 every 8 connection(s)
I20260619 15:01:37.663385 32179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/data/info.pb
I20260619 15:01:37.668681 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 32179
I20260619 15:01:37.668891 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/wal/instance
I20260619 15:01:37.669490 11967 external_mini_cluster.cc:1428] Setting key 148f5def918f4101385d14bad64a1ad4
I20260619 15:01:37.678835 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.666315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.195:59829 (local address 127.11.175.254:36955)
0619 15:01:37.666533 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:37.666542 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:37.668363 (+ 1821us) server_negotiation.cc:407] Connection header received
0619 15:01:37.670809 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:37.670818 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:37.670913 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:37.671044 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:37.672724 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.673610 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.675610 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.676137 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.676258 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.677900 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:37.677906 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:37.677910 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:37.677937 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:37.677973 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:37.677986 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:37.678119 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:37.678363 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 15:01:37.678454 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 15:01:37.679999 32308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.666963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:36955 (local address 127.11.175.195:59829)
0619 15:01:37.668051 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:37.668149 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:37.670433 (+ 2284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:37.671339 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:37.671348 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:37.671514 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:01:37.672223 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.672264 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.673851 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.673855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:37.675434 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.675456 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.676390 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.676396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:37.676639 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.677515 (+ 876us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 15:01:37.677518 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:01:37.677566 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 15:01:37.677667 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:01:37.678116 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:01:37.678126 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:01:37.678132 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 15:01:37.678330 (+ 198us) client_negotiation.cc:240] Negotiation successful
0619 15:01:37.678757 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":458,"threads_started":1}
I20260619 15:01:37.682235 32306 heartbeater.cc:347] Connected to a master server at 127.11.175.254:36955
I20260619 15:01:37.682596 32306 heartbeater.cc:464] Registering TS with master...
I20260619 15:01:37.683585 32306 heartbeater.cc:511] Master 127.11.175.254:36955 requested a full tablet report, sending...
I20260619 15:01:37.685400 31858 ts_manager.cc:194] Registered new tserver with Master: d7cf49afafb644ac9c26751d0c0ab773 (127.11.175.195:33331)
I20260619 15:01:37.686241 31858 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.175.195:59829
I20260619 15:01:37.698915 11967 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 15:01:37.710291 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.703625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59644 (local address 127.11.175.254:36955)
0619 15:01:37.703877 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:37.703882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:37.704364 (+ 482us) server_negotiation.cc:407] Connection header received
0619 15:01:37.704493 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:37.704497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:37.704571 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:37.704674 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:37.705310 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.706062 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.707710 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.708190 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.708314 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.709537 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:37.709542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:37.709545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:37.709566 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:37.709602 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:37.709615 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:37.709707 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:37.709838 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 15:01:37.709998 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260619 15:01:37.718241 32045 heartbeater.cc:503] Master 127.11.175.254:36955 was elected leader, sending a full tablet report...
I20260619 15:01:37.714927 31858 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 15:01:37.722548 31858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 15:01:37.751435 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.735820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:33331 (local address 127.0.0.1:60378)
0619 15:01:37.737966 (+ 2146us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:37.738005 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:37.738178 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:37.744119 (+ 5941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:37.744128 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:37.744164 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:37.744655 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.744676 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.746704 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.746710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:37.748723 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.748736 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.750497 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.750503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:37.750772 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.750848 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 15:01:37.750949 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 15:01:37.751107 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1848,"mutex_wait_us":159,"thread_start_us":835,"threads_started":1}
I20260619 15:01:37.752717 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.736135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60378 (local address 127.11.175.195:33331)
0619 15:01:37.743369 (+ 7234us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:37.743378 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:37.743413 (+ 35us) server_negotiation.cc:407] Connection header received
0619 15:01:37.743523 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:37.743531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:37.743825 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:37.744600 (+ 775us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:37.744900 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.746520 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.748942 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.750273 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.752013 (+ 1740us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.752168 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:37.752225 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 15:01:37.752357 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7049,"thread_start_us":6892,"threads_started":1}
I20260619 15:01:37.752795 32318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.735880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:37991 (local address 127.0.0.1:43342)
0619 15:01:37.738489 (+ 2609us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:37.738505 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:37.738606 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:37.744705 (+ 6099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:37.744711 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:37.744746 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:37.745163 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.745175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.747154 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.747159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:37.749626 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.749638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.751909 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.751916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:37.752088 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.752130 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 15:01:37.752204 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 15:01:37.752351 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2491,"mutex_wait_us":72,"spinlock_wait_cycles":1280,"thread_start_us":1670,"threads_started":1}
I20260619 15:01:37.753048 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.737942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38299 (local address 127.0.0.1:38062)
0619 15:01:37.742592 (+ 4650us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:37.742610 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:37.743357 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:37.744427 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:37.744434 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:37.744465 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:37.745012 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.745028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.746850 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.746855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:37.749587 (+ 2732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.749604 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.752234 (+ 2630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.752240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:37.752386 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.752414 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 15:01:37.752503 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 15:01:37.752641 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4523,"mutex_wait_us":94,"thread_start_us":4371,"threads_started":1}
I20260619 15:01:37.753151 32323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.738515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38062 (local address 127.11.175.193:38299)
0619 15:01:37.742249 (+ 3734us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:37.742259 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:37.742832 (+ 573us) server_negotiation.cc:407] Connection header received
0619 15:01:37.743559 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:37.743566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:37.743818 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:37.744007 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:37.745219 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.746676 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.749838 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.751589 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.751830 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.752089 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:37.752589 (+ 500us) server_negotiation.cc:299] Negotiation successful
0619 15:01:37.752791 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3539,"thread_start_us":3397,"threads_started":1}
I20260619 15:01:37.754318 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.736432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43342 (local address 127.11.175.194:37991)
0619 15:01:37.744158 (+ 7726us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:37.744165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:37.744188 (+ 23us) server_negotiation.cc:407] Connection header received
0619 15:01:37.744273 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:37.744278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:37.744447 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:37.745296 (+ 849us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:37.745525 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.747006 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.749846 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.751589 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.753533 (+ 1944us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:37.753692 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:37.753762 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 15:01:37.753949 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7574,"thread_start_us":7440,"threads_started":1}
I20260619 15:01:37.757310 31980 tablet_service.cc:1467] Processing CreateTablet for tablet 482821b29bce45bab8ecf53c91a207a8 (DEFAULT_TABLE table=test-table [id=0bf591be4e134c5083126970476b9ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:37.757310 32241 tablet_service.cc:1467] Processing CreateTablet for tablet 482821b29bce45bab8ecf53c91a207a8 (DEFAULT_TABLE table=test-table [id=0bf591be4e134c5083126970476b9ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:37.757720 32111 tablet_service.cc:1467] Processing CreateTablet for tablet 482821b29bce45bab8ecf53c91a207a8 (DEFAULT_TABLE table=test-table [id=0bf591be4e134c5083126970476b9ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 15:01:37.758945 32241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 482821b29bce45bab8ecf53c91a207a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:37.758945 31980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 482821b29bce45bab8ecf53c91a207a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:37.759052 32111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 482821b29bce45bab8ecf53c91a207a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:37.771678 32324 tablet_bootstrap.cc:492] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: Bootstrap starting.
I20260619 15:01:37.776887 32324 tablet_bootstrap.cc:654] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:37.778226 32325 tablet_bootstrap.cc:492] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d: Bootstrap starting.
I20260619 15:01:37.778613 32324 log.cc:824] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:37.779281 32326 tablet_bootstrap.cc:492] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: Bootstrap starting.
I20260619 15:01:37.782249 32324 tablet_bootstrap.cc:492] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: No bootstrap required, opened a new log
I20260619 15:01:37.782702 32324 ts_tablet_manager.cc:1397] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260619 15:01:37.783445 32326 tablet_bootstrap.cc:654] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:37.783725 32325 tablet_bootstrap.cc:654] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:37.784884 32325 log.cc:824] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:37.784885 32326 log.cc:824] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:37.788470 32325 tablet_bootstrap.cc:492] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d: No bootstrap required, opened a new log
I20260619 15:01:37.788473 32326 tablet_bootstrap.cc:492] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: No bootstrap required, opened a new log
I20260619 15:01:37.788801 32326 ts_tablet_manager.cc:1397] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260619 15:01:37.788800 32325 ts_tablet_manager.cc:1397] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.006s
I20260619 15:01:37.791193 32324 raft_consensus.cc:348] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 } } peers { permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 } } peers { permanent_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38299 } }
I20260619 15:01:37.791606 32324 raft_consensus.cc:374] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:37.791772 32324 raft_consensus.cc:729] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0307672fbd3c4ee3b0536f9e90306a77, State: Initialized, Role: FOLLOWER
I20260619 15:01:37.792418 32324 consensus_queue.cc:260] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 } } peers { permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 } } peers { permanent_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38299 } }
I20260619 15:01:37.795607 32176 heartbeater.cc:503] Master 127.11.175.254:36955 was elected leader, sending a full tablet report...
I20260619 15:01:37.796376 32324 ts_tablet_manager.cc:1428] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260619 15:01:37.800155 32325 raft_consensus.cc:348] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 } } peers { permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 } } peers { permanent_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38299 } }
I20260619 15:01:37.800770 32325 raft_consensus.cc:374] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:37.800406 32326 raft_consensus.cc:348] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 } } peers { permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 } } peers { permanent_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38299 } }
I20260619 15:01:37.800894 32325 raft_consensus.cc:729] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58460e7dc7224cbd9eec338c6a2aa62d, State: Initialized, Role: FOLLOWER
I20260619 15:01:37.800949 32326 raft_consensus.cc:374] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:37.801040 32326 raft_consensus.cc:729] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7cf49afafb644ac9c26751d0c0ab773, State: Initialized, Role: FOLLOWER
I20260619 15:01:37.801688 32326 consensus_queue.cc:260] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 } } peers { permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 } } peers { permanent_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38299 } }
I20260619 15:01:37.801699 32325 consensus_queue.cc:260] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 } } peers { permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 } } peers { permanent_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38299 } }
I20260619 15:01:37.803318 32306 heartbeater.cc:503] Master 127.11.175.254:36955 was elected leader, sending a full tablet report...
I20260619 15:01:37.804015 32325 ts_tablet_manager.cc:1428] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260619 15:01:37.805140 32326 ts_tablet_manager.cc:1428] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: Time spent starting tablet: real 0.016s user 0.008s sys 0.008s
W20260619 15:01:37.915154 32307 tablet.cc:2369] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 15:01:37.939702 32177 tablet.cc:2369] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 15:01:37.942447 32046 tablet.cc:2369] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:01:37.992566 32331 raft_consensus.cc:482] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 15:01:37.992919 32331 raft_consensus.cc:504] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 } } peers { permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 } } peers { permanent_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38299 } }
I20260619 15:01:37.994621 32331 leader_election.cc:288] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7cf49afafb644ac9c26751d0c0ab773 (127.11.175.195:33331), 0307672fbd3c4ee3b0536f9e90306a77 (127.11.175.194:37991)
I20260619 15:01:38.003922 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.995192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:54661 (local address 127.11.175.195:33331)
0619 15:01:37.995444 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:37.995452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:37.995992 (+ 540us) server_negotiation.cc:407] Connection header received
0619 15:01:37.996117 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:37.996123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:37.996205 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:37.996322 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:37.997166 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.998228 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.001403 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.002723 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.002890 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:38.002960 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:38.003536 (+ 576us) server_negotiation.cc:299] Negotiation successful
0619 15:01:38.003701 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260619 15:01:38.003912 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.995313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.175.193:52175 (local address 127.11.175.194:37991)
0619 15:01:37.995535 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:37.995541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:37.995892 (+ 351us) server_negotiation.cc:407] Connection header received
0619 15:01:37.996118 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:37.996123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:37.996277 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:37.996369 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 15:01:37.997087 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.998228 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.001403 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.002745 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.002893 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:38.002960 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:38.003501 (+ 541us) server_negotiation.cc:299] Negotiation successful
0619 15:01:38.003638 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 15:01:38.004096 32334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.994982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.195:33331 (local address 127.11.175.193:54661)
0619 15:01:37.995882 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:37.995903 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:37.996000 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:37.996362 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:37.996368 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:37.996396 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:37.996992 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.997005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.998373 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.998379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:38.001154 (+ 2775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:38.001170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.002966 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.002971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:38.003195 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:38.003308 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 15:01:38.003405 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 15:01:38.003632 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":576,"thread_start_us":372,"threads_started":1}
I20260619 15:01:38.004211 32333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:37.994982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.194:37991 (local address 127.11.175.193:52175)
0619 15:01:37.995762 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:37.995784 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 15:01:37.995960 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:01:37.996386 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:01:37.996389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:01:37.996404 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 15:01:37.996905 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:37.996926 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:37.998373 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:37.998379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:38.001154 (+ 2775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:01:38.001170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.002951 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.002959 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:01:38.003201 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:38.003304 (+ 103us) client_negotiation.cc:769] Sending connection context
0619 15:01:38.003405 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 15:01:38.003632 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":276,"thread_start_us":373,"threads_started":1}
I20260619 15:01:38.005481 32131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "482821b29bce45bab8ecf53c91a207a8" candidate_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0307672fbd3c4ee3b0536f9e90306a77" is_pre_election: true
I20260619 15:01:38.005481 32261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "482821b29bce45bab8ecf53c91a207a8" candidate_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7cf49afafb644ac9c26751d0c0ab773" is_pre_election: true
I20260619 15:01:38.006125 32261 raft_consensus.cc:2454] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58460e7dc7224cbd9eec338c6a2aa62d in term 0.
I20260619 15:01:38.006157 32131 raft_consensus.cc:2454] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58460e7dc7224cbd9eec338c6a2aa62d in term 0.
I20260619 15:01:38.006991 31934 leader_election.cc:302] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58460e7dc7224cbd9eec338c6a2aa62d, d7cf49afafb644ac9c26751d0c0ab773; no voters:
I20260619 15:01:38.007524 32331 raft_consensus.cc:2781] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 15:01:38.007681 32331 raft_consensus.cc:482] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 15:01:38.007818 32331 raft_consensus.cc:3037] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:38.010083 32331 raft_consensus.cc:504] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 } } peers { permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 } } peers { permanent_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38299 } }
I20260619 15:01:38.010838 32331 leader_election.cc:288] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [CANDIDATE]: Term 1 election: Requested vote from peers d7cf49afafb644ac9c26751d0c0ab773 (127.11.175.195:33331), 0307672fbd3c4ee3b0536f9e90306a77 (127.11.175.194:37991)
I20260619 15:01:38.011405 32261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "482821b29bce45bab8ecf53c91a207a8" candidate_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7cf49afafb644ac9c26751d0c0ab773"
I20260619 15:01:38.011405 32131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "482821b29bce45bab8ecf53c91a207a8" candidate_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0307672fbd3c4ee3b0536f9e90306a77"
I20260619 15:01:38.011631 32261 raft_consensus.cc:3037] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:38.011650 32131 raft_consensus.cc:3037] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:38.014953 32261 raft_consensus.cc:2454] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58460e7dc7224cbd9eec338c6a2aa62d in term 1.
I20260619 15:01:38.014953 32131 raft_consensus.cc:2454] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58460e7dc7224cbd9eec338c6a2aa62d in term 1.
I20260619 15:01:38.015509 31935 leader_election.cc:302] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0307672fbd3c4ee3b0536f9e90306a77, 58460e7dc7224cbd9eec338c6a2aa62d; no voters:
I20260619 15:01:38.015880 32331 raft_consensus.cc:2781] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:38.016409 32331 raft_consensus.cc:686] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 1 LEADER]: Becoming Leader. State: Replica: 58460e7dc7224cbd9eec338c6a2aa62d, State: Running, Role: LEADER
I20260619 15:01:38.016909 32331 consensus_queue.cc:237] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 } } peers { permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 } } peers { permanent_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38299 } }
I20260619 15:01:38.021950 31858 catalog_manager.cc:5495] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d reported cstate change: term changed from 0 to 1, leader changed from <none> to 58460e7dc7224cbd9eec338c6a2aa62d (127.11.175.193). New cstate: current_term: 1 leader_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58460e7dc7224cbd9eec338c6a2aa62d" member_type: VOTER last_known_addr { host: "127.11.175.193" port: 38299 } health_report { overall_health: HEALTHY } } }
I20260619 15:01:38.053236 32323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.047521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38074 (local address 127.11.175.193:38299)
0619 15:01:38.047709 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:38.047714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:38.047819 (+ 105us) server_negotiation.cc:407] Connection header received
0619 15:01:38.047948 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:38.047952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:38.048046 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:38.048219 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:01:38.048707 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.049454 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.051078 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.051521 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.051638 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:38.052175 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:01:38.052714 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:01:38.052834 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:38.052960 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 15:01:38.053031 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 15:01:38.062518 32261 raft_consensus.cc:1261] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773 [term 1 FOLLOWER]: Refusing update from remote peer 58460e7dc7224cbd9eec338c6a2aa62d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:38.062505 32131 raft_consensus.cc:1261] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77 [term 1 FOLLOWER]: Refusing update from remote peer 58460e7dc7224cbd9eec338c6a2aa62d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:01:38.063570 32335 consensus_queue.cc:1035] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7cf49afafb644ac9c26751d0c0ab773" member_type: VOTER last_known_addr { host: "127.11.175.195" port: 33331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:38.064133 32331 consensus_queue.cc:1035] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0307672fbd3c4ee3b0536f9e90306a77" member_type: VOTER last_known_addr { host: "127.11.175.194" port: 37991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:01:38.077181 32338 mvcc.cc:205] Tried to move back new op lower bound from 7298585796852989952 to 7298585796680949760. Current Snapshot: MvccSnapshot[applied={T|T < 7298585796852989952}]
I20260619 15:01:38.078611 32340 mvcc.cc:205] Tried to move back new op lower bound from 7298585796852989952 to 7298585796680949760. Current Snapshot: MvccSnapshot[applied={T|T < 7298585796852989952}]
I20260619 15:01:38.079478 32339 mvcc.cc:205] Tried to move back new op lower bound from 7298585796852989952 to 7298585796680949760. Current Snapshot: MvccSnapshot[applied={T|T < 7298585796852989952}]
I20260619 15:01:38.092430 31859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:38.092762 31859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 15:01:38.097081 31859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 035169628b8944d9b3d37156bc0f8e9d: Sending DeleteTablet for 3 replicas of tablet 482821b29bce45bab8ecf53c91a207a8
I20260619 15:01:38.098459 32241 tablet_service.cc:1514] Processing DeleteTablet for tablet 482821b29bce45bab8ecf53c91a207a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:38 UTC) from {username='slave'} at 127.0.0.1:60378
I20260619 15:01:38.098747 32111 tablet_service.cc:1514] Processing DeleteTablet for tablet 482821b29bce45bab8ecf53c91a207a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:38 UTC) from {username='slave'} at 127.0.0.1:43342
I20260619 15:01:38.098852 31980 tablet_service.cc:1514] Processing DeleteTablet for tablet 482821b29bce45bab8ecf53c91a207a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 15:01:38 UTC) from {username='slave'} at 127.0.0.1:38062
I20260619 15:01:38.104317 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 31918
I20260619 15:01:38.112223 32349 tablet_replica.cc:330] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d: stopping tablet replica
I20260619 15:01:38.112999 32349 raft_consensus.cc:2229] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 1 LEADER]: Raft consensus shutting down.
I20260619 15:01:38.113116 32348 tablet_replica.cc:330] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: stopping tablet replica
I20260619 15:01:38.113551 32348 raft_consensus.cc:2229] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:38.114321 32348 raft_consensus.cc:2258] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:38.122787 32348 ts_tablet_manager.cc:1905] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:38.131172 32347 tablet_replica.cc:330] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: stopping tablet replica
I20260619 15:01:38.131220 32349 raft_consensus.cc:2258] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:38.131808 32347 raft_consensus.cc:2229] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 15:01:38.132371 32347 raft_consensus.cc:2258] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 15:01:38.134617 32323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.106001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.11.175.193:38299)
0619 15:01:38.106213 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:38.106220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:38.114823 (+ 8603us) server_negotiation.cc:407] Connection header received
0619 15:01:38.115119 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:38.115125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:38.115249 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:38.115367 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:38.116156 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.117644 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.120021 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.120788 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.120907 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:38.132471 (+ 11564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:38.132487 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:38.132507 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:38.133388 (+ 881us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:38.133458 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:38.133490 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:38.133623 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:38.134059 (+ 436us) server_negotiation.cc:299] Negotiation successful
0619 15:01:38.134197 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 15:01:38.134817 32347 ts_tablet_manager.cc:1905] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:38.137378 32040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:38.139539 32348 ts_tablet_manager.cc:1918] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:38.139792 32348 log.cc:1196] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-1/wal/wals/482821b29bce45bab8ecf53c91a207a8
I20260619 15:01:38.140501 32348 ts_tablet_manager.cc:1939] T 482821b29bce45bab8ecf53c91a207a8 P 0307672fbd3c4ee3b0536f9e90306a77: Deleting consensus metadata
I20260619 15:01:38.142004 32347 ts_tablet_manager.cc:1918] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 15:01:38.142241 32347 log.cc:1196] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: Deleting WAL directory at /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781880972997616-11967-0/minicluster-data/ts-2/wal/wals/482821b29bce45bab8ecf53c91a207a8
I20260619 15:01:38.143031 31845 catalog_manager.cc:4841] TS 0307672fbd3c4ee3b0536f9e90306a77 (127.11.175.194:37991): tablet 482821b29bce45bab8ecf53c91a207a8 (table test-table [id=0bf591be4e134c5083126970476b9ebd]) successfully deleted
I20260619 15:01:38.143527 32347 ts_tablet_manager.cc:1939] T 482821b29bce45bab8ecf53c91a207a8 P d7cf49afafb644ac9c26751d0c0ab773: Deleting consensus metadata
I20260619 15:01:38.145444 31844 catalog_manager.cc:4841] TS d7cf49afafb644ac9c26751d0c0ab773 (127.11.175.195:33331): tablet 482821b29bce45bab8ecf53c91a207a8 (table test-table [id=0bf591be4e134c5083126970476b9ebd]) successfully deleted
I20260619 15:01:38.138703 32349 ts_tablet_manager.cc:1905] T 482821b29bce45bab8ecf53c91a207a8 P 58460e7dc7224cbd9eec338c6a2aa62d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 15:01:38.246840 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 31918
I20260619 15:01:38.270051 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 32049
W20260619 15:01:38.269874 31845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.175.193:38299 (error 108)
I20260619 15:01:38.272744 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.271764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38299 (local address 127.0.0.1:38086)
0619 15:01:38.272396 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:38.272574 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":364,"threads_started":1}
W20260619 15:01:38.273021 31845 catalog_manager.cc:4584] TS 58460e7dc7224cbd9eec338c6a2aa62d (127.11.175.193:38299): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 482821b29bce45bab8ecf53c91a207a8: Network error: Client connection negotiation failed: client connection to 127.11.175.193:38299: connect: Connection refused (error 111)
I20260619 15:01:38.279314 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.270980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.11.175.194:37991)
0619 15:01:38.271306 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:38.271311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:38.271331 (+ 20us) server_negotiation.cc:407] Connection header received
0619 15:01:38.271633 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:38.271638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:38.271738 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:38.271901 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:38.272446 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.273471 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.275839 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.276395 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.276517 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:38.277853 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:38.277862 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:38.277876 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:38.278495 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:38.278547 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:38.278568 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:38.278753 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:38.278903 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 15:01:38.279003 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260619 15:01:38.280018 32171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:38.326997 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.326362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38299 (local address 127.0.0.1:38088)
0619 15:01:38.326657 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:38.326820 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260619 15:01:38.328234 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.327792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38299 (local address 127.0.0.1:38092)
0619 15:01:38.327971 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:38.328068 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 15:01:38.372398 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 32049
I20260619 15:01:38.389230 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.388607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38299 (local address 127.0.0.1:38102)
0619 15:01:38.388882 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:38.389008 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260619 15:01:38.390512 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.390008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38299 (local address 127.0.0.1:38106)
0619 15:01:38.390205 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:38.390305 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 15:01:38.390702 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 32179
I20260619 15:01:38.397709 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.391460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.11.175.195:33331)
0619 15:01:38.391619 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:38.391623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:38.391641 (+ 18us) server_negotiation.cc:407] Connection header received
0619 15:01:38.391748 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:38.391751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:38.391843 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:38.391931 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:38.392338 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.393058 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.394366 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:38.395024 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:38.395126 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:38.396282 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:38.396290 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:38.396304 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:38.396912 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:38.396972 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:38.396992 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:38.397077 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:38.397355 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 15:01:38.397452 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 15:01:38.398429 32301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:38.471141 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.470591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38299 (local address 127.0.0.1:38120)
0619 15:01:38.470880 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:38.470981 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260619 15:01:38.472441 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:38.471977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.193:38299 (local address 127.0.0.1:38130)
0619 15:01:38.472187 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 15:01:38.472303 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.193:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260619 15:01:38.488739 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 32179
I20260619 15:01:38.505764 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 31828
I20260619 15:01:38.507045 31889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:38.621497 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 31828
2026-06-19T15:01:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3055 ms)
[----------] 6 tests from EncryptionPolicyTest (22975 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:38 dist-test-slave-76rk krb5kdc[32359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:38 dist-test-slave-76rk krb5kdc[32359](info): set up 2 sockets
Jun 19 15:01:38 dist-test-slave-76rk krb5kdc[32359](info): commencing operation
krb5kdc: starting...
W20260619 15:01:39.767418 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:39 dist-test-slave-76rk krb5kdc[32359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:01:40.061100 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:40.418349 32372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:40.418808 32372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:40.418987 32372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:40.426623 32372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:40.426832 32372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:40.426930 32372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:40.427018 32372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:40.427073 32372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:40.438145 32372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35177
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:40.439580 32372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:40.440959 32372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:40.453899 32377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:40.453933 32378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:40.453804 32380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:40.454955 32372 server_base.cc:1034] running on GCE node
I20260619 15:01:40.455775 32372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:40.457454 32372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:40.458719 32372 hybrid_clock.cc:648] HybridClock initialized: now 1781881300458664 us; error 44 us; skew 500 ppm
I20260619 15:01:40.459131 32372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:40 dist-test-slave-76rk krb5kdc[32359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881300, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:40.462456 32372 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:01:40.464131 32372 webserver.cc:460] Webserver started at http://127.0.0.1:36835/ using document root <none> and password file <none>
I20260619 15:01:40.464808 32372 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:40.464915 32372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:40.465168 32372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:40.468365 32372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "9d37ca10ca2e40e6bcddde9fcdffa858"
format_stamp: "Formatted at 2026-06-19 15:01:40 on dist-test-slave-76rk"
server_key: "626dab0f22d232db44688e668a7f03da"
server_key_iv: "17f40c10dfb4e5294593795742babffb"
server_key_version: "encryptionkey@0"
I20260619 15:01:40.469223 32372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "9d37ca10ca2e40e6bcddde9fcdffa858"
format_stamp: "Formatted at 2026-06-19 15:01:40 on dist-test-slave-76rk"
server_key: "626dab0f22d232db44688e668a7f03da"
server_key_iv: "17f40c10dfb4e5294593795742babffb"
server_key_version: "encryptionkey@0"
I20260619 15:01:40.474949 32372 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 15:01:40.478668 32387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:40.480288 32372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 15:01:40.480487 32372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "9d37ca10ca2e40e6bcddde9fcdffa858"
format_stamp: "Formatted at 2026-06-19 15:01:40 on dist-test-slave-76rk"
server_key: "626dab0f22d232db44688e668a7f03da"
server_key_iv: "17f40c10dfb4e5294593795742babffb"
server_key_version: "encryptionkey@0"
I20260619 15:01:40.480687 32372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:40.510481 32372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:40.515321 32372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:40.515574 32372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:40.535547 32372 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35177
I20260619 15:01:40.535580 32438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35177 every 8 connection(s)
I20260619 15:01:40.537005 32372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:40.541041 32439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:40.542419 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 32372
I20260619 15:01:40.542668 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:40.543527 11967 external_mini_cluster.cc:1428] Setting key 4847812508f818f16e42a44ca05529f0
I20260619 15:01:40.554971 32439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858: Bootstrap starting.
I20260619 15:01:40.558290 32439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858: Neither blocks nor log segments found. Creating new log.
Jun 19 15:01:40 dist-test-slave-76rk krb5kdc[32359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 15:01:40.559520 32439 log.cc:824] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:40.562847 32439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858: No bootstrap required, opened a new log
I20260619 15:01:40.569839 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:40.545817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56655 (local address 127.0.0.1:35177)
0619 15:01:40.546795 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:40.546807 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:40.546857 (+ 50us) server_negotiation.cc:407] Connection header received
0619 15:01:40.548650 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:40.548679 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:40.549150 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:40.549569 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:40.551468 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:40.552983 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:40.554469 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:40.555393 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:40.555630 (+ 237us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:40.560610 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:40.560622 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:40.560645 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:40.560695 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:40.564384 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:40.565460 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:40.565468 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:40.565476 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:40.565690 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:40.566284 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:40.566288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:40.566294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:40.567137 (+ 843us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:40.567549 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:40.568210 (+ 661us) server_negotiation.cc:299] Negotiation successful
0619 15:01:40.568634 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":387,"threads_started":1}
I20260619 15:01:40.571947 32439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d37ca10ca2e40e6bcddde9fcdffa858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35177 } }
I20260619 15:01:40.572383 32439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:40.572536 32439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d37ca10ca2e40e6bcddde9fcdffa858, State: Initialized, Role: FOLLOWER
I20260619 15:01:40.573238 32439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d37ca10ca2e40e6bcddde9fcdffa858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35177 } }
I20260619 15:01:40.573520 32439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:40.573684 32439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:40.573908 32439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:40.574978 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:01:40.576538 32439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d37ca10ca2e40e6bcddde9fcdffa858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35177 } }
I20260619 15:01:40.577140 32439 leader_election.cc:302] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d37ca10ca2e40e6bcddde9fcdffa858; no voters:
I20260619 15:01:40.577876 32439 leader_election.cc:288] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:40.578001 32444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:40.581830 32444 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [term 1 LEADER]: Becoming Leader. State: Replica: 9d37ca10ca2e40e6bcddde9fcdffa858, State: Running, Role: LEADER
I20260619 15:01:40.582563 32444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d37ca10ca2e40e6bcddde9fcdffa858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35177 } }
I20260619 15:01:40.583206 32439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:40.593429 32447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d37ca10ca2e40e6bcddde9fcdffa858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d37ca10ca2e40e6bcddde9fcdffa858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35177 } } }
I20260619 15:01:40.594398 32447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:40.594600 32448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d37ca10ca2e40e6bcddde9fcdffa858. Latest consensus state: current_term: 1 leader_uuid: "9d37ca10ca2e40e6bcddde9fcdffa858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d37ca10ca2e40e6bcddde9fcdffa858" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35177 } } }
I20260619 15:01:40.595438 32448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:40.600744 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:40.580363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58993 (local address 127.0.0.1:35177)
0619 15:01:40.580643 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:40.580651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:40.580970 (+ 319us) server_negotiation.cc:407] Connection header received
0619 15:01:40.581297 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:40.581302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:40.581377 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:40.581446 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:40.583483 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:40.584619 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:40.586189 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:40.586722 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:40.586873 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:40.589623 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:40.589628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:40.589632 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:40.589670 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:40.594026 (+ 4356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:40.595714 (+ 1688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:40.595723 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:40.595729 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:40.595939 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:40.596881 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:40.596889 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:40.596895 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:40.597554 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:40.599631 (+ 2077us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:40.599901 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 15:01:40.600176 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260619 15:01:40.602418 32456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:40.608927 32456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:40.618727 32456 catalog_manager.cc:1347] Generated new cluster ID: b163868c793c4987b022957ec1a5a8ea
I20260619 15:01:40.618870 32456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:40.650157 32456 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:40.651729 32456 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:40.659041 32456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d37ca10ca2e40e6bcddde9fcdffa858: Generated new TSK 0
I20260619 15:01:40.660036 32456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:40.667939 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 32372
I20260619 15:01:40.669375 32434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:40.764725 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 32372
W20260619 15:01:40.783718 32357 connection.cc:535] client connection to 127.0.0.1:35177 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35177): Connection reset by peer (error 104)
2026-06-19T15:01:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2134 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:40 dist-test-slave-76rk krb5kdc[32471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:40 dist-test-slave-76rk krb5kdc[32471](info): set up 2 sockets
Jun 19 15:01:40 dist-test-slave-76rk krb5kdc[32471](info): commencing operation
krb5kdc: starting...
W20260619 15:01:41.899292 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.008s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:42 dist-test-slave-76rk krb5kdc[32471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:01:42.191953 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43465
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:42.541359 32484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:42.541687 32484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:42.541859 32484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:42.549660 32484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:42.549782 32484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:42.549871 32484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:42.549970 32484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:42.550066 32484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:42.560576 32484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:42.562052 32484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:42.563988 32484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:42.576740 32490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:42.576869 32489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:42.576833 32492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:42.578024 32484 server_base.cc:1034] running on GCE node
I20260619 15:01:42.579201 32484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:42.580818 32484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:42.582086 32484 hybrid_clock.cc:648] HybridClock initialized: now 1781881302582037 us; error 41 us; skew 500 ppm
I20260619 15:01:42.582509 32484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:42 dist-test-slave-76rk krb5kdc[32471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881302, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:42.585971 32484 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:01:42.587680 32484 webserver.cc:460] Webserver started at http://127.0.0.1:38107/ using document root <none> and password file <none>
I20260619 15:01:42.588409 32484 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:42.588563 32484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:42.588940 32484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:42.592128 32484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "b46aa444d3ac4493ac68dc65c17e16a5"
format_stamp: "Formatted at 2026-06-19 15:01:42 on dist-test-slave-76rk"
server_key: "65b61ada840867dcf4b5647840c34221"
server_key_iv: "06af8c4ec1d7f247eddccd365ad54725"
server_key_version: "encryptionkey@0"
I20260619 15:01:42.592953 32484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "b46aa444d3ac4493ac68dc65c17e16a5"
format_stamp: "Formatted at 2026-06-19 15:01:42 on dist-test-slave-76rk"
server_key: "65b61ada840867dcf4b5647840c34221"
server_key_iv: "06af8c4ec1d7f247eddccd365ad54725"
server_key_version: "encryptionkey@0"
I20260619 15:01:42.598763 32484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260619 15:01:42.602860 32499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:42.604427 32484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 15:01:42.604642 32484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "b46aa444d3ac4493ac68dc65c17e16a5"
format_stamp: "Formatted at 2026-06-19 15:01:42 on dist-test-slave-76rk"
server_key: "65b61ada840867dcf4b5647840c34221"
server_key_iv: "06af8c4ec1d7f247eddccd365ad54725"
server_key_version: "encryptionkey@0"
I20260619 15:01:42.604867 32484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:42.618980 32484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:42.624037 32484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:42.624387 32484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:42.645608 32484 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43465
I20260619 15:01:42.645655 32550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43465 every 8 connection(s)
I20260619 15:01:42.647213 32484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:42.650429 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 32484
I20260619 15:01:42.650709 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:42.651211 11967 external_mini_cluster.cc:1428] Setting key 4f9c30f0ae224df6de9f4e526ae9680b
I20260619 15:01:42.651953 32551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:42.666615 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:42.653171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43969 (local address 127.0.0.1:43465)
0619 15:01:42.654070 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:42.654083 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:42.654127 (+ 44us) server_negotiation.cc:407] Connection header received
0619 15:01:42.655882 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:42.655908 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:42.656389 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:42.656766 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:42.657547 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:42.659145 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:42.661264 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:42.662050 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:42.662220 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:42.663732 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:42.663741 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:42.663762 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:42.664460 (+ 698us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:42.664551 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:42.664583 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:42.664728 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:42.665318 (+ 590us) server_negotiation.cc:299] Negotiation successful
0619 15:01:42.665544 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":416,"threads_started":1}
I20260619 15:01:42.667390 32551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5: Bootstrap starting.
I20260619 15:01:42.670821 32551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:42.670861 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:01:42.672137 32551 log.cc:824] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:42.675856 32551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5: No bootstrap required, opened a new log
I20260619 15:01:42.682596 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:42.676218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56053 (local address 127.0.0.1:43465)
0619 15:01:42.676454 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:42.676461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:42.676751 (+ 290us) server_negotiation.cc:407] Connection header received
0619 15:01:42.676980 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:42.676985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:42.677045 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:42.677157 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:42.677737 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:42.678559 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:42.679913 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:42.680436 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:42.680575 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:42.681796 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:42.681803 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:42.681807 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:42.681826 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:42.681870 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:42.681888 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:42.682030 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:42.682219 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 15:01:42.682287 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 15:01:42.684254 32551 raft_consensus.cc:348] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46aa444d3ac4493ac68dc65c17e16a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } }
I20260619 15:01:42.684737 32551 raft_consensus.cc:374] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:42.684868 32551 raft_consensus.cc:729] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46aa444d3ac4493ac68dc65c17e16a5, State: Initialized, Role: FOLLOWER
I20260619 15:01:42.685454 32551 consensus_queue.cc:260] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46aa444d3ac4493ac68dc65c17e16a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } }
I20260619 15:01:42.685707 32551 raft_consensus.cc:388] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:42.685859 32551 raft_consensus.cc:482] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:42.686045 32551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:42.688663 32551 raft_consensus.cc:504] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46aa444d3ac4493ac68dc65c17e16a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } }
I20260619 15:01:42.689369 32551 leader_election.cc:302] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b46aa444d3ac4493ac68dc65c17e16a5; no voters:
I20260619 15:01:42.690208 32551 leader_election.cc:288] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:42.690383 32561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:42.691205 32561 raft_consensus.cc:686] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [term 1 LEADER]: Becoming Leader. State: Replica: b46aa444d3ac4493ac68dc65c17e16a5, State: Running, Role: LEADER
I20260619 15:01:42.691890 32561 consensus_queue.cc:237] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46aa444d3ac4493ac68dc65c17e16a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } }
I20260619 15:01:42.692551 32551 sys_catalog.cc:564] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:42.695362 32562 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b46aa444d3ac4493ac68dc65c17e16a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46aa444d3ac4493ac68dc65c17e16a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } } }
I20260619 15:01:42.695879 32562 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:42.695387 32563 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b46aa444d3ac4493ac68dc65c17e16a5. Latest consensus state: current_term: 1 leader_uuid: "b46aa444d3ac4493ac68dc65c17e16a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46aa444d3ac4493ac68dc65c17e16a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } } }
I20260619 15:01:42.696596 32563 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:42.697690 32570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:42.703460 32570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:42.715518 32570 catalog_manager.cc:1347] Generated new cluster ID: 2b9d2d496abd48bba0e22208f874eb9f
I20260619 15:01:42.715673 32570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:42.743528 32570 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:42.745033 32570 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:42.754514 32570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b46aa444d3ac4493ac68dc65c17e16a5: Generated new TSK 0
I20260619 15:01:42.755467 32570 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:42.775537 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 32484
W20260619 15:01:42.777487 32546 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43969
W20260619 15:01:42.778390 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 32484: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:01:42.778592 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 32484
W20260619 15:01:42.796025 32469 connection.cc:535] client connection to 127.0.0.1:43465 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43465): Connection reset by peer (error 104)
2026-06-19T15:01:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2011 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:42 dist-test-slave-76rk krb5kdc[32582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:42 dist-test-slave-76rk krb5kdc[32582](info): set up 2 sockets
Jun 19 15:01:42 dist-test-slave-76rk krb5kdc[32582](info): commencing operation
krb5kdc: starting...
W20260619 15:01:43.915807 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:44 dist-test-slave-76rk krb5kdc[32582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:01:44.203107 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:44.557396 32595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:44.557736 32595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:44.557832 32595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:44.565688 32595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:44.565825 32595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:44.565873 32595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:44.565939 32595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:44.565985 32595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:44.577068 32595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42405
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:44.578404 32595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:44.579763 32595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:44.592782 32601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:44.592722 32600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:44.592684 32603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:44.594079 32595 server_base.cc:1034] running on GCE node
I20260619 15:01:44.595091 32595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:44.596814 32595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:44.598116 32595 hybrid_clock.cc:648] HybridClock initialized: now 1781881304598057 us; error 54 us; skew 500 ppm
I20260619 15:01:44.598621 32595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:44 dist-test-slave-76rk krb5kdc[32582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881304, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:44.607323 32595 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:01:44.609148 32595 webserver.cc:460] Webserver started at http://127.0.0.1:39689/ using document root <none> and password file <none>
I20260619 15:01:44.610085 32595 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:44.610272 32595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:44.610620 32595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:44.614069 32595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda"
format_stamp: "Formatted at 2026-06-19 15:01:44 on dist-test-slave-76rk"
server_key: "d73c93a13c9e206a17e58682830ca688"
server_key_iv: "358458c8cb01ebe6e54f72ea22770b57"
server_key_version: "encryptionkey@0"
I20260619 15:01:44.615034 32595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda"
format_stamp: "Formatted at 2026-06-19 15:01:44 on dist-test-slave-76rk"
server_key: "d73c93a13c9e206a17e58682830ca688"
server_key_iv: "358458c8cb01ebe6e54f72ea22770b57"
server_key_version: "encryptionkey@0"
I20260619 15:01:44.621016 32595 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260619 15:01:44.625515 32610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:44.627122 32595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 15:01:44.627344 32595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda"
format_stamp: "Formatted at 2026-06-19 15:01:44 on dist-test-slave-76rk"
server_key: "d73c93a13c9e206a17e58682830ca688"
server_key_iv: "358458c8cb01ebe6e54f72ea22770b57"
server_key_version: "encryptionkey@0"
I20260619 15:01:44.627581 32595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:44.634163 32595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:44.639359 32595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:44.639705 32595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:44.660112 32595 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42405
I20260619 15:01:44.660151 32661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42405 every 8 connection(s)
I20260619 15:01:44.661674 32595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:44.662988 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 32595
I20260619 15:01:44.663194 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:44.663731 11967 external_mini_cluster.cc:1428] Setting key fd16b98b16b40a403dcfaca8a9268ca2
I20260619 15:01:44.666723 32662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:44.672174 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:44.665591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35457 (local address 127.0.0.1:42405)
0619 15:01:44.666371 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:44.666384 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:44.666427 (+ 43us) server_negotiation.cc:407] Connection header received
0619 15:01:44.668231 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:44.668255 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:44.668625 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:44.669058 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:44.670186 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:44.670192 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:44.670205 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:44.670220 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:44.670305 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:44.670336 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:44.670580 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:44.670765 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 15:01:44.671134 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":326,"threads_started":1}
I20260619 15:01:44.676386 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:01:44.679330 32662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda: Bootstrap starting.
I20260619 15:01:44.683532 32662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:44.684026 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:44.681207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53369 (local address 127.0.0.1:42405)
0619 15:01:44.681427 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:44.681435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:44.681856 (+ 421us) server_negotiation.cc:407] Connection header received
0619 15:01:44.682038 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:44.682044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:44.682117 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:44.682237 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:44.683314 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:44.683322 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:44.683330 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:44.683353 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:44.683413 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:44.683439 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:44.683579 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:44.683631 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 15:01:44.683692 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 15:01:44.684855 32662 log.cc:824] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:44.688027 32662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda: No bootstrap required, opened a new log
I20260619 15:01:44.695739 32662 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } }
I20260619 15:01:44.696194 32662 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:44.696326 32662 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bb0433dc0bf4b73b4b34c5ddb121eda, State: Initialized, Role: FOLLOWER
I20260619 15:01:44.697037 32662 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } }
I20260619 15:01:44.697321 32662 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:44.697481 32662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:44.697669 32662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:44.700063 32662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } }
I20260619 15:01:44.700685 32662 leader_election.cc:302] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bb0433dc0bf4b73b4b34c5ddb121eda; no voters:
I20260619 15:01:44.701483 32662 leader_election.cc:288] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:44.701707 32672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:44.702905 32672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [term 1 LEADER]: Becoming Leader. State: Replica: 1bb0433dc0bf4b73b4b34c5ddb121eda, State: Running, Role: LEADER
I20260619 15:01:44.703490 32672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } }
I20260619 15:01:44.703877 32662 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:44.706524 32674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bb0433dc0bf4b73b4b34c5ddb121eda. Latest consensus state: current_term: 1 leader_uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } } }
I20260619 15:01:44.707171 32674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:44.706561 32673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bb0433dc0bf4b73b4b34c5ddb121eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } } }
I20260619 15:01:44.708148 32673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:44.712127 32680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:44.718007 32680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:44.728847 32680 catalog_manager.cc:1347] Generated new cluster ID: b8cee8f1515b4e11b6776036d5de4db3
I20260619 15:01:44.729008 32680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:44.749743 32680 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:44.750381 32680 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:44.758301 32680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bb0433dc0bf4b73b4b34c5ddb121eda: Generated new TSK 0
I20260619 15:01:44.759351 32680 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:44.769505 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 32595
W20260619 15:01:44.771289 32657 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35457
W20260619 15:01:44.772203 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 32595: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:01:44.772359 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 32595
2026-06-19T15:01:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1991 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:44 dist-test-slave-76rk krb5kdc[32694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:44 dist-test-slave-76rk krb5kdc[32694](info): set up 2 sockets
Jun 19 15:01:44 dist-test-slave-76rk krb5kdc[32694](info): commencing operation
krb5kdc: starting...
W20260619 15:01:45.923240 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:46 dist-test-slave-76rk krb5kdc[32694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:01:46.215083 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:46.569667 32707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:46.570055 32707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:46.570225 32707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:46.578162 32707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:46.578343 32707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:46.578457 32707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:46.578563 32707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:46.578675 32707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:46.589113 32707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:46.590509 32707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:46.591953 32707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:46.606711 32713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:46.607241 32715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:46.607775 32712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:46.608285 32707 server_base.cc:1034] running on GCE node
I20260619 15:01:46.609184 32707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:46.610850 32707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:46.612110 32707 hybrid_clock.cc:648] HybridClock initialized: now 1781881306612048 us; error 58 us; skew 500 ppm
I20260619 15:01:46.612527 32707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:46 dist-test-slave-76rk krb5kdc[32694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881306, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:46.616320 32707 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:01:46.618043 32707 webserver.cc:460] Webserver started at http://127.0.0.1:40509/ using document root <none> and password file <none>
I20260619 15:01:46.618870 32707 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:46.619021 32707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:46.619342 32707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:46.622520 32707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "79ebbd3781a04a28bd5f38d5f753eda9"
format_stamp: "Formatted at 2026-06-19 15:01:46 on dist-test-slave-76rk"
server_key: "fb901a5eedd439ecec48b92a25858c20"
server_key_iv: "6b561ac62a27bc3f811f09e44f47e9e4"
server_key_version: "encryptionkey@0"
I20260619 15:01:46.623452 32707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "79ebbd3781a04a28bd5f38d5f753eda9"
format_stamp: "Formatted at 2026-06-19 15:01:46 on dist-test-slave-76rk"
server_key: "fb901a5eedd439ecec48b92a25858c20"
server_key_iv: "6b561ac62a27bc3f811f09e44f47e9e4"
server_key_version: "encryptionkey@0"
I20260619 15:01:46.629356 32707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 15:01:46.633744 32722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:46.635751 32707 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260619 15:01:46.636003 32707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "79ebbd3781a04a28bd5f38d5f753eda9"
format_stamp: "Formatted at 2026-06-19 15:01:46 on dist-test-slave-76rk"
server_key: "fb901a5eedd439ecec48b92a25858c20"
server_key_iv: "6b561ac62a27bc3f811f09e44f47e9e4"
server_key_version: "encryptionkey@0"
I20260619 15:01:46.636257 32707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:46.666738 32707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:46.671257 32707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:46.671557 32707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:46.692886 32707 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41333
I20260619 15:01:46.692919 305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41333 every 8 connection(s)
I20260619 15:01:46.694824 32707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:46.699177 306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:46.704937 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 32707
I20260619 15:01:46.705147 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:46.705801 11967 external_mini_cluster.cc:1428] Setting key d1ba3074c7fe13c6c66293000fafa60a
I20260619 15:01:46.711306 306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9: Bootstrap starting.
I20260619 15:01:46.715484 306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:46.717000 306 log.cc:824] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:46.721253 306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9: No bootstrap required, opened a new log
Jun 19 15:01:46 dist-test-slave-76rk krb5kdc[32694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 15:01:46.731505 306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ebbd3781a04a28bd5f38d5f753eda9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } }
I20260619 15:01:46.731990 306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:46.732118 306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ebbd3781a04a28bd5f38d5f753eda9, State: Initialized, Role: FOLLOWER
I20260619 15:01:46.732878 306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ebbd3781a04a28bd5f38d5f753eda9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } }
I20260619 15:01:46.733176 306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:46.733326 306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:46.733510 306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:46.734855 309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:46.708195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58867 (local address 127.0.0.1:41333)
0619 15:01:46.709233 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:46.709246 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:46.709292 (+ 46us) server_negotiation.cc:407] Connection header received
0619 15:01:46.711240 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:46.711268 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:46.711756 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:46.712128 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:46.714281 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:46.715841 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:46.717836 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:46.719059 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:46.719232 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:46.725287 (+ 6055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:46.725297 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:46.725320 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:46.725373 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:46.729314 (+ 3941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:46.730447 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:46.730455 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:46.730462 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:46.730743 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:46.731230 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:46.731233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:46.731235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:46.731867 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:46.732200 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:46.732754 (+ 554us) server_negotiation.cc:299] Negotiation successful
0619 15:01:46.733284 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":800,"thread_start_us":474,"threads_started":1}
I20260619 15:01:46.736539 306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ebbd3781a04a28bd5f38d5f753eda9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } }
I20260619 15:01:46.737144 306 leader_election.cc:302] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ebbd3781a04a28bd5f38d5f753eda9; no voters:
I20260619 15:01:46.737805 306 leader_election.cc:288] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:46.737927 311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:46.739279 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:01:46.740103 306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:46.742985 311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [term 1 LEADER]: Becoming Leader. State: Replica: 79ebbd3781a04a28bd5f38d5f753eda9, State: Running, Role: LEADER
I20260619 15:01:46.743754 311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ebbd3781a04a28bd5f38d5f753eda9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } }
I20260619 15:01:46.747969 313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ebbd3781a04a28bd5f38d5f753eda9. Latest consensus state: current_term: 1 leader_uuid: "79ebbd3781a04a28bd5f38d5f753eda9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ebbd3781a04a28bd5f38d5f753eda9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } } }
I20260619 15:01:46.748442 313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:46.749008 312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ebbd3781a04a28bd5f38d5f753eda9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ebbd3781a04a28bd5f38d5f753eda9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } } }
I20260619 15:01:46.749290 312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:46.753350 326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:46.759572 326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:46.774459 309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:46.748652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49299 (local address 127.0.0.1:41333)
0619 15:01:46.750389 (+ 1737us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:46.750399 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:46.751771 (+ 1372us) server_negotiation.cc:407] Connection header received
0619 15:01:46.752015 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:46.752025 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:46.752161 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:46.752335 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:46.755541 (+ 3206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:46.756813 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:46.758875 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:46.759644 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:46.759938 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:46.763685 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:46.763698 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:46.763703 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:46.763755 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:46.768018 (+ 4263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:46.769678 (+ 1660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:46.769684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:46.769687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:46.769890 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:46.770529 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:46.770537 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:46.770543 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:46.771207 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:46.773194 (+ 1987us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:46.773424 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 15:01:46.773794 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1533}
I20260619 15:01:46.779726 326 catalog_manager.cc:1347] Generated new cluster ID: 5461a6b566244fa6bbdd31254e515e31
I20260619 15:01:46.779865 326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:46.802281 326 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:46.803740 326 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:46.814924 326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ebbd3781a04a28bd5f38d5f753eda9: Generated new TSK 0
I20260619 15:01:46.815915 326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:46.826124 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 32707
I20260619 15:01:46.827426 301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:46.927901 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 32707
2026-06-19T15:01:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2158 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:47 dist-test-slave-76rk krb5kdc[338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:47 dist-test-slave-76rk krb5kdc[338](info): set up 2 sockets
Jun 19 15:01:47 dist-test-slave-76rk krb5kdc[338](info): commencing operation
krb5kdc: starting...
W20260619 15:01:48.111275 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:48 dist-test-slave-76rk krb5kdc[338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:01:48.394963 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:48.748497 351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:48.748871 351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:48.749032 351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:48.756779 351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:48.756969 351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:48.757081 351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:48.757190 351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:48.757273 351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:48.768265 351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:48.769732 351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:48.771221 351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:48.784605 357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:48.784749 359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:48.784996 356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:48.786500 351 server_base.cc:1034] running on GCE node
I20260619 15:01:48.787696 351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:48.789435 351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:48.790777 351 hybrid_clock.cc:648] HybridClock initialized: now 1781881308790643 us; error 122 us; skew 500 ppm
I20260619 15:01:48.791427 351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:48 dist-test-slave-76rk krb5kdc[338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881308, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:48.795905 351 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:01:48.797977 351 webserver.cc:460] Webserver started at http://127.0.0.1:44827/ using document root <none> and password file <none>
I20260619 15:01:48.798861 351 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:48.799019 351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:48.799350 351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:48.802795 351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "4c1c994727324df0b1f77a0c6bf9299c"
format_stamp: "Formatted at 2026-06-19 15:01:48 on dist-test-slave-76rk"
server_key: "42d12f1de8ed65cedc9c32eaf8fd371b"
server_key_iv: "0d56dc415b7246d979e40de15a30dd0f"
server_key_version: "encryptionkey@0"
I20260619 15:01:48.803665 351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "4c1c994727324df0b1f77a0c6bf9299c"
format_stamp: "Formatted at 2026-06-19 15:01:48 on dist-test-slave-76rk"
server_key: "42d12f1de8ed65cedc9c32eaf8fd371b"
server_key_iv: "0d56dc415b7246d979e40de15a30dd0f"
server_key_version: "encryptionkey@0"
I20260619 15:01:48.809762 351 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260619 15:01:48.814749 366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:48.816679 351 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260619 15:01:48.816958 351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "4c1c994727324df0b1f77a0c6bf9299c"
format_stamp: "Formatted at 2026-06-19 15:01:48 on dist-test-slave-76rk"
server_key: "42d12f1de8ed65cedc9c32eaf8fd371b"
server_key_iv: "0d56dc415b7246d979e40de15a30dd0f"
server_key_version: "encryptionkey@0"
I20260619 15:01:48.817209 351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:48.844911 351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:48.849781 351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:48.850106 351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:48.877393 351 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41207
I20260619 15:01:48.877496 417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41207 every 8 connection(s)
I20260619 15:01:48.879089 351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:48.883770 418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:48.884222 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 351
I20260619 15:01:48.884434 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:48.884984 11967 external_mini_cluster.cc:1428] Setting key 68fb0537c2c74fe4f6b618c0d2d71d31
I20260619 15:01:48.899261 418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c: Bootstrap starting.
I20260619 15:01:48.900991 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:48.887228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37665 (local address 127.0.0.1:41207)
0619 15:01:48.888179 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:48.888192 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:48.888237 (+ 45us) server_negotiation.cc:407] Connection header received
0619 15:01:48.891010 (+ 2773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:48.891050 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:48.891553 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:48.891943 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:48.892623 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:48.893991 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:48.895986 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:48.896748 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:48.896915 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:48.898320 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:48.898330 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:48.898346 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:48.899014 (+ 668us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:48.899105 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:48.899138 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:48.899237 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:48.899574 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 15:01:48.899920 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":463,"threads_started":1}
I20260619 15:01:48.902908 418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:48.904033 418 log.cc:824] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:48.905613 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:01:48.907512 418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c: No bootstrap required, opened a new log
I20260619 15:01:48.918447 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:48.911199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56915 (local address 127.0.0.1:41207)
0619 15:01:48.911393 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:48.911401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:48.911918 (+ 517us) server_negotiation.cc:407] Connection header received
0619 15:01:48.912094 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:48.912102 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:48.912200 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:48.912327 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:48.912895 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:48.914054 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:48.915623 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:48.916357 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:48.916495 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:48.917675 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:48.917680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:48.917684 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:48.917707 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:48.917767 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:48.917788 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:48.917878 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:48.918036 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 15:01:48.918145 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 15:01:48.918114 418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1c994727324df0b1f77a0c6bf9299c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260619 15:01:48.918696 418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:48.918802 418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1c994727324df0b1f77a0c6bf9299c, State: Initialized, Role: FOLLOWER
I20260619 15:01:48.919503 418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1c994727324df0b1f77a0c6bf9299c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260619 15:01:48.919798 418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:48.919977 418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:48.920156 418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:48.922478 418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1c994727324df0b1f77a0c6bf9299c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260619 15:01:48.923039 418 leader_election.cc:302] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c1c994727324df0b1f77a0c6bf9299c; no voters:
I20260619 15:01:48.923702 418 leader_election.cc:288] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:48.923905 428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:48.925055 428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [term 1 LEADER]: Becoming Leader. State: Replica: 4c1c994727324df0b1f77a0c6bf9299c, State: Running, Role: LEADER
I20260619 15:01:48.925619 428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1c994727324df0b1f77a0c6bf9299c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260619 15:01:48.925916 418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:48.928390 430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c1c994727324df0b1f77a0c6bf9299c. Latest consensus state: current_term: 1 leader_uuid: "4c1c994727324df0b1f77a0c6bf9299c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1c994727324df0b1f77a0c6bf9299c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } } }
I20260619 15:01:48.928797 430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:48.928735 429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c1c994727324df0b1f77a0c6bf9299c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1c994727324df0b1f77a0c6bf9299c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } } }
I20260619 15:01:48.929232 429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:48.930281 436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:48.936961 436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:48.946580 436 catalog_manager.cc:1347] Generated new cluster ID: 2d5d0064274a4c688d9378501b43bbcb
I20260619 15:01:48.946797 436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:48.960412 436 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:48.962466 436 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:48.975106 436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c1c994727324df0b1f77a0c6bf9299c: Generated new TSK 0
I20260619 15:01:48.976383 436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:48.985680 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 351
W20260619 15:01:48.987298 413 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37665
W20260619 15:01:48.988129 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 351: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:01:48.988310 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 351
2026-06-19T15:01:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2056 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:49 dist-test-slave-76rk krb5kdc[449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:49 dist-test-slave-76rk krb5kdc[449](info): set up 2 sockets
Jun 19 15:01:49 dist-test-slave-76rk krb5kdc[449](info): commencing operation
krb5kdc: starting...
W20260619 15:01:50.127480 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:50 dist-test-slave-76rk krb5kdc[449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:01:50.431635 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33915
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:50.787757 462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:50.788166 462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:50.788446 462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:50.796613 462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:50.796818 462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:50.796918 462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:50.797032 462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:50.797119 462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:50.807741 462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:50.809193 462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:50.810694 462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:50.825229 470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:50.825346 467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:50.825205 468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:50.826342 462 server_base.cc:1034] running on GCE node
I20260619 15:01:50.827613 462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:50.829396 462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:50.830756 462 hybrid_clock.cc:648] HybridClock initialized: now 1781881310830586 us; error 78 us; skew 500 ppm
I20260619 15:01:50.831221 462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:50 dist-test-slave-76rk krb5kdc[449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881310, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:50.841739 462 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:01:50.843792 462 webserver.cc:460] Webserver started at http://127.0.0.1:44091/ using document root <none> and password file <none>
I20260619 15:01:50.844605 462 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:50.844754 462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:50.845136 462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:50.848485 462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "3690d64848b0455baf9ae3082e3fca4a"
format_stamp: "Formatted at 2026-06-19 15:01:50 on dist-test-slave-76rk"
server_key: "135eb520bf5670453aae7759ac1c4fe0"
server_key_iv: "2d34d21956c58b6c5e8c9b2f9cf9c498"
server_key_version: "encryptionkey@0"
I20260619 15:01:50.849375 462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "3690d64848b0455baf9ae3082e3fca4a"
format_stamp: "Formatted at 2026-06-19 15:01:50 on dist-test-slave-76rk"
server_key: "135eb520bf5670453aae7759ac1c4fe0"
server_key_iv: "2d34d21956c58b6c5e8c9b2f9cf9c498"
server_key_version: "encryptionkey@0"
I20260619 15:01:50.855443 462 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 15:01:50.859812 477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:50.861420 462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260619 15:01:50.861680 462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "3690d64848b0455baf9ae3082e3fca4a"
format_stamp: "Formatted at 2026-06-19 15:01:50 on dist-test-slave-76rk"
server_key: "135eb520bf5670453aae7759ac1c4fe0"
server_key_iv: "2d34d21956c58b6c5e8c9b2f9cf9c498"
server_key_version: "encryptionkey@0"
I20260619 15:01:50.862038 462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:50.872809 462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:50.878180 462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:50.878544 462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:50.900144 462 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33915
I20260619 15:01:50.900239 528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33915 every 8 connection(s)
I20260619 15:01:50.901727 462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:50.905912 529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:50.910904 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 462
I20260619 15:01:50.911132 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:50.911615 11967 external_mini_cluster.cc:1428] Setting key 39749f0a957c5a6f10845d73863665ca
I20260619 15:01:50.917775 529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a: Bootstrap starting.
I20260619 15:01:50.920133 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:50.913472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33047 (local address 127.0.0.1:33915)
0619 15:01:50.914380 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:50.914396 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:50.914444 (+ 48us) server_negotiation.cc:407] Connection header received
0619 15:01:50.916321 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:50.916348 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:50.916761 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:50.917139 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:50.918374 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:50.918381 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:50.918394 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:50.918414 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:50.918482 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:50.918503 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:50.918761 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:50.918875 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 15:01:50.919096 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":419,"threads_started":1}
I20260619 15:01:50.921772 529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:50.923101 529 log.cc:824] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:50.924564 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:01:50.926545 529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a: No bootstrap required, opened a new log
I20260619 15:01:50.932864 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:50.930213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58009 (local address 127.0.0.1:33915)
0619 15:01:50.930390 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:50.930394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:50.930689 (+ 295us) server_negotiation.cc:407] Connection header received
0619 15:01:50.930848 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:50.930851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:50.930907 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:50.930994 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:50.932002 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:50.932006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:50.932010 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:50.932030 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:50.932118 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:50.932138 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:50.932241 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:50.932423 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 15:01:50.932483 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 15:01:50.936709 529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3690d64848b0455baf9ae3082e3fca4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260619 15:01:50.937134 529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:50.937251 529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3690d64848b0455baf9ae3082e3fca4a, State: Initialized, Role: FOLLOWER
I20260619 15:01:50.937860 529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3690d64848b0455baf9ae3082e3fca4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260619 15:01:50.938110 529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:50.938257 529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:50.938439 529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:50.941341 529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3690d64848b0455baf9ae3082e3fca4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260619 15:01:50.941895 529 leader_election.cc:302] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3690d64848b0455baf9ae3082e3fca4a; no voters:
I20260619 15:01:50.942503 529 leader_election.cc:288] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:50.942709 539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:50.943435 539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [term 1 LEADER]: Becoming Leader. State: Replica: 3690d64848b0455baf9ae3082e3fca4a, State: Running, Role: LEADER
I20260619 15:01:50.944049 539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3690d64848b0455baf9ae3082e3fca4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } }
I20260619 15:01:50.944794 529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:50.947170 540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3690d64848b0455baf9ae3082e3fca4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3690d64848b0455baf9ae3082e3fca4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } } }
I20260619 15:01:50.947707 540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:50.947247 541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3690d64848b0455baf9ae3082e3fca4a. Latest consensus state: current_term: 1 leader_uuid: "3690d64848b0455baf9ae3082e3fca4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3690d64848b0455baf9ae3082e3fca4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33915 } } }
I20260619 15:01:50.948037 541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:50.950929 547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:50.955564 547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:50.964740 547 catalog_manager.cc:1347] Generated new cluster ID: a7ff39a80cc548f28fcb682606bad317
I20260619 15:01:50.964885 547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:51.007930 547 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:51.008498 547 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:51.019927 547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3690d64848b0455baf9ae3082e3fca4a: Generated new TSK 0
I20260619 15:01:51.020896 547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:51.029994 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 462
W20260619 15:01:51.031426 524 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33047
W20260619 15:01:51.032300 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 462: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:01:51.032477 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 462
2026-06-19T15:01:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2051 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:51 dist-test-slave-76rk krb5kdc[560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:51 dist-test-slave-76rk krb5kdc[560](info): set up 2 sockets
Jun 19 15:01:51 dist-test-slave-76rk krb5kdc[560](info): commencing operation
krb5kdc: starting...
W20260619 15:01:52.184823 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:52 dist-test-slave-76rk krb5kdc[560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:01:52.486075 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:52.848394 573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:52.848809 573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:52.848989 573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:52.856717 573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:52.856930 573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:52.857048 573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:52.857156 573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:52.857241 573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:52.872720 573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:52.874209 573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:52.875756 573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:52.889832 578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:52.889866 579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:52.890074 581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:52.890659 573 server_base.cc:1034] running on GCE node
I20260619 15:01:52.891599 573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:52.893260 573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:52.894527 573 hybrid_clock.cc:648] HybridClock initialized: now 1781881312894487 us; error 41 us; skew 500 ppm
I20260619 15:01:52.895013 573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:52 dist-test-slave-76rk krb5kdc[560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881312, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:52.898759 573 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:01:52.900617 573 webserver.cc:460] Webserver started at http://127.0.0.1:38751/ using document root <none> and password file <none>
I20260619 15:01:52.901441 573 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:52.901623 573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:52.902076 573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:52.905782 573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "b3ed9fcecfc640589d808282cba0edc5"
format_stamp: "Formatted at 2026-06-19 15:01:52 on dist-test-slave-76rk"
server_key: "a8c2216d03976ff5442487e49aa1b196"
server_key_iv: "e98b534c284cb8ac66efee276abef95d"
server_key_version: "encryptionkey@0"
I20260619 15:01:52.906831 573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "b3ed9fcecfc640589d808282cba0edc5"
format_stamp: "Formatted at 2026-06-19 15:01:52 on dist-test-slave-76rk"
server_key: "a8c2216d03976ff5442487e49aa1b196"
server_key_iv: "e98b534c284cb8ac66efee276abef95d"
server_key_version: "encryptionkey@0"
I20260619 15:01:52.913661 573 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 15:01:52.918002 588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:52.919829 573 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:01:52.920068 573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "b3ed9fcecfc640589d808282cba0edc5"
format_stamp: "Formatted at 2026-06-19 15:01:52 on dist-test-slave-76rk"
server_key: "a8c2216d03976ff5442487e49aa1b196"
server_key_iv: "e98b534c284cb8ac66efee276abef95d"
server_key_version: "encryptionkey@0"
I20260619 15:01:52.920312 573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:52.951061 573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:52.956205 573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:52.956574 573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:52.977548 573 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41429
I20260619 15:01:52.977658 639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41429 every 8 connection(s)
I20260619 15:01:52.979118 573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:52.983686 640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:52.986312 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 573
I20260619 15:01:52.986567 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:52.987262 11967 external_mini_cluster.cc:1428] Setting key 82e80b4729bd45df6e0eadceb08b9bbc
I20260619 15:01:52.999423 640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5: Bootstrap starting.
I20260619 15:01:53.003985 640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5: Neither blocks nor log segments found. Creating new log.
Jun 19 15:01:53 dist-test-slave-76rk krb5kdc[560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 15:01:53.005476 640 log.cc:824] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:53.010125 640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5: No bootstrap required, opened a new log
I20260619 15:01:53.017767 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:52.989869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:41379 (local address 127.0.0.1:41429)
0619 15:01:52.990828 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:52.990842 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:52.990896 (+ 54us) server_negotiation.cc:407] Connection header received
0619 15:01:52.992794 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:52.992821 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:52.993452 (+ 631us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:52.994002 (+ 550us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:52.996053 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:52.997533 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:52.999570 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:53.000410 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:53.000596 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:53.005989 (+ 5393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:53.005999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:53.006017 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:53.006063 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:53.011241 (+ 5178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:53.012567 (+ 1326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:53.012577 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:53.012587 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:53.013052 (+ 465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:53.013685 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:53.013691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:53.013695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:53.014722 (+ 1027us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:53.015130 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:53.015699 (+ 569us) server_negotiation.cc:299] Negotiation successful
0619 15:01:53.016127 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":433,"threads_started":1}
I20260619 15:01:53.022015 640 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3ed9fcecfc640589d808282cba0edc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41429 } }
I20260619 15:01:53.022643 640 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:53.022835 640 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ed9fcecfc640589d808282cba0edc5, State: Initialized, Role: FOLLOWER
I20260619 15:01:53.023281 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:01:53.023576 640 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3ed9fcecfc640589d808282cba0edc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41429 } }
I20260619 15:01:53.023885 640 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:53.024041 640 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:53.024207 640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:53.028039 640 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3ed9fcecfc640589d808282cba0edc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41429 } }
I20260619 15:01:53.028872 640 leader_election.cc:302] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3ed9fcecfc640589d808282cba0edc5; no voters:
I20260619 15:01:53.029660 640 leader_election.cc:288] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:53.029860 649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:53.030825 649 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [term 1 LEADER]: Becoming Leader. State: Replica: b3ed9fcecfc640589d808282cba0edc5, State: Running, Role: LEADER
I20260619 15:01:53.031629 649 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3ed9fcecfc640589d808282cba0edc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41429 } }
I20260619 15:01:53.033866 649 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3ed9fcecfc640589d808282cba0edc5. Latest consensus state: current_term: 1 leader_uuid: "b3ed9fcecfc640589d808282cba0edc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3ed9fcecfc640589d808282cba0edc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41429 } } }
I20260619 15:01:53.034317 649 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:53.034785 652 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3ed9fcecfc640589d808282cba0edc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3ed9fcecfc640589d808282cba0edc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41429 } } }
I20260619 15:01:53.035231 652 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:53.038470 640 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:53.043882 654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:53.051945 654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:53.052928 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:53.029582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:60655 (local address 127.0.0.1:41429)
0619 15:01:53.029815 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:53.029822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:53.030791 (+ 969us) server_negotiation.cc:407] Connection header received
0619 15:01:53.031149 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:53.031159 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:53.031270 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:53.031363 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:53.034306 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:53.036040 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:53.038109 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:53.038888 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:53.039006 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:53.043508 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:53.043514 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:53.043518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:53.043557 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:53.047535 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:53.048827 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:53.048833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:53.048837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:53.049037 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:53.049899 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:53.049907 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:53.049911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:53.050378 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:53.050729 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:53.052265 (+ 1536us) server_negotiation.cc:299] Negotiation successful
0619 15:01:53.052486 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 15:01:53.066301 654 catalog_manager.cc:1347] Generated new cluster ID: 8026ead72eda476088287df777f71c6d
I20260619 15:01:53.066445 654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:53.131641 654 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:53.133634 654 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:53.146870 654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3ed9fcecfc640589d808282cba0edc5: Generated new TSK 0
I20260619 15:01:53.147822 654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:53.163130 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 573
I20260619 15:01:53.164520 635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:53.265434 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 573
W20260619 15:01:53.284210 558 connection.cc:535] client connection to 127.0.0.1:41429 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41429): Connection reset by peer (error 104)
2026-06-19T15:01:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2229 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:53 dist-test-slave-76rk krb5kdc[672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:53 dist-test-slave-76rk krb5kdc[672](info): set up 2 sockets
Jun 19 15:01:53 dist-test-slave-76rk krb5kdc[672](info): commencing operation
krb5kdc: starting...
W20260619 15:01:54.403280 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:54 dist-test-slave-76rk krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:01:54.690519 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:55.057439 685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:55.057775 685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:55.057883 685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:55.068394 685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:55.068581 685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:55.068678 685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:55.068799 685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:55.068883 685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:55.081807 685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:55.083343 685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:55.084769 685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:55.098140 693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:55.098160 691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:55.098287 690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:55.104964 685 server_base.cc:1034] running on GCE node
I20260619 15:01:55.105978 685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:55.107929 685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:55.109289 685 hybrid_clock.cc:648] HybridClock initialized: now 1781881315109179 us; error 102 us; skew 500 ppm
I20260619 15:01:55.110122 685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:55 dist-test-slave-76rk krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881315, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:55.113909 685 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:01:55.115962 685 webserver.cc:460] Webserver started at http://127.0.0.1:34257/ using document root <none> and password file <none>
I20260619 15:01:55.116808 685 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:55.116930 685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:55.117208 685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:55.120625 685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "fc34b707d2164a07a9573df645454ca2"
format_stamp: "Formatted at 2026-06-19 15:01:55 on dist-test-slave-76rk"
server_key: "22c95b4c00d3334aa88e7c88271cb2c6"
server_key_iv: "0ba26e0bb364b21a13e6101aeff6cfaa"
server_key_version: "encryptionkey@0"
I20260619 15:01:55.121469 685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "fc34b707d2164a07a9573df645454ca2"
format_stamp: "Formatted at 2026-06-19 15:01:55 on dist-test-slave-76rk"
server_key: "22c95b4c00d3334aa88e7c88271cb2c6"
server_key_iv: "0ba26e0bb364b21a13e6101aeff6cfaa"
server_key_version: "encryptionkey@0"
I20260619 15:01:55.127828 685 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260619 15:01:55.131892 700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:55.133623 685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 15:01:55.133845 685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "fc34b707d2164a07a9573df645454ca2"
format_stamp: "Formatted at 2026-06-19 15:01:55 on dist-test-slave-76rk"
server_key: "22c95b4c00d3334aa88e7c88271cb2c6"
server_key_iv: "0ba26e0bb364b21a13e6101aeff6cfaa"
server_key_version: "encryptionkey@0"
I20260619 15:01:55.134049 685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:55.155435 685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:55.160813 685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:55.161186 685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:55.182673 685 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41497
I20260619 15:01:55.182775 751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41497 every 8 connection(s)
I20260619 15:01:55.184175 685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:55.188138 752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:55.190104 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 685
I20260619 15:01:55.190321 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:55.190917 11967 external_mini_cluster.cc:1428] Setting key 08e371662af9196082a456a20d3698ec
I20260619 15:01:55.202395 752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2: Bootstrap starting.
I20260619 15:01:55.205121 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:55.193118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:35399 (local address 127.0.0.1:41497)
0619 15:01:55.194021 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:55.194038 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:55.194082 (+ 44us) server_negotiation.cc:407] Connection header received
0619 15:01:55.195906 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:55.195933 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:55.196435 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:55.196881 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:55.197482 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:55.198884 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:55.200285 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:55.201042 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:55.201255 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:55.202626 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:55.202693 (+ 67us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:55.202721 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:55.203321 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:55.203404 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:55.203430 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:55.203612 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:55.203907 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 15:01:55.204098 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":401,"threads_started":1}
I20260619 15:01:55.206058 752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:55.207275 752 log.cc:824] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:55.209453 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:01:55.210711 752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2: No bootstrap required, opened a new log
I20260619 15:01:55.219043 752 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc34b707d2164a07a9573df645454ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41497 } }
I20260619 15:01:55.219498 752 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:55.219656 752 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc34b707d2164a07a9573df645454ca2, State: Initialized, Role: FOLLOWER
I20260619 15:01:55.220438 752 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc34b707d2164a07a9573df645454ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41497 } }
I20260619 15:01:55.220647 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:55.214125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:47021 (local address 127.0.0.1:41497)
0619 15:01:55.214353 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:55.214360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:55.214695 (+ 335us) server_negotiation.cc:407] Connection header received
0619 15:01:55.214947 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:55.214952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:55.215016 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:55.215136 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:55.215661 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:55.216457 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:55.217822 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:55.218272 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:55.218411 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:55.219687 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:55.219695 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:55.219700 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:55.219726 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:55.219800 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:55.219825 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:55.219948 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:55.220211 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 15:01:55.220284 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 15:01:55.220798 752 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:55.220965 752 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:55.221181 752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:55.223737 752 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc34b707d2164a07a9573df645454ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41497 } }
I20260619 15:01:55.224282 752 leader_election.cc:302] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc34b707d2164a07a9573df645454ca2; no voters:
I20260619 15:01:55.224829 752 leader_election.cc:288] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:55.224999 762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:55.225845 762 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [term 1 LEADER]: Becoming Leader. State: Replica: fc34b707d2164a07a9573df645454ca2, State: Running, Role: LEADER
I20260619 15:01:55.226451 762 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc34b707d2164a07a9573df645454ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41497 } }
I20260619 15:01:55.227603 752 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:55.230947 764 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc34b707d2164a07a9573df645454ca2. Latest consensus state: current_term: 1 leader_uuid: "fc34b707d2164a07a9573df645454ca2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc34b707d2164a07a9573df645454ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41497 } } }
I20260619 15:01:55.231381 764 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:55.232339 763 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc34b707d2164a07a9573df645454ca2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc34b707d2164a07a9573df645454ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41497 } } }
I20260619 15:01:55.232762 763 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:55.235294 770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:55.241487 770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:55.252348 770 catalog_manager.cc:1347] Generated new cluster ID: 60b64ec1c01b4fb3a77a21c314526084
I20260619 15:01:55.252527 770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:55.284621 770 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:55.286111 770 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:55.292855 770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc34b707d2164a07a9573df645454ca2: Generated new TSK 0
I20260619 15:01:55.293761 770 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:55.307824 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 685
W20260619 15:01:55.309067 747 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.48:35399
W20260619 15:01:55.309991 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 685: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:01:55.310209 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 685
W20260619 15:01:55.325186 670 connection.cc:535] client connection to 127.0.0.1:41497 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41497): Connection reset by peer (error 104)
2026-06-19T15:01:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2040 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:55 dist-test-slave-76rk krb5kdc[783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:55 dist-test-slave-76rk krb5kdc[783](info): set up 2 sockets
Jun 19 15:01:55 dist-test-slave-76rk krb5kdc[783](info): commencing operation
krb5kdc: starting...
W20260619 15:01:56.495409 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:56 dist-test-slave-76rk krb5kdc[783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:01:56.788458 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:57.147558 796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:57.147879 796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:57.147980 796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:57.155651 796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:57.155783 796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:57.155830 796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:57.155886 796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:57.155930 796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:57.166363 796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:57.167793 796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:57.169098 796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:57.182243 801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:57.182307 802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:57.182288 804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:57.184042 796 server_base.cc:1034] running on GCE node
I20260619 15:01:57.184957 796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:57.186599 796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:57.187961 796 hybrid_clock.cc:648] HybridClock initialized: now 1781881317187902 us; error 52 us; skew 500 ppm
I20260619 15:01:57.188460 796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:57 dist-test-slave-76rk krb5kdc[783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881317, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:57.197098 796 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:01:57.199178 796 webserver.cc:460] Webserver started at http://127.0.0.1:35361/ using document root <none> and password file <none>
I20260619 15:01:57.199954 796 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:57.200085 796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:57.200410 796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:57.204447 796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "7f80ab09958c40c9bdb5bec14342fe87"
format_stamp: "Formatted at 2026-06-19 15:01:57 on dist-test-slave-76rk"
server_key: "afea55361844083e54aadac3c39b2819"
server_key_iv: "5c668c7d44ae98a466802b6f16b0cfd5"
server_key_version: "encryptionkey@0"
I20260619 15:01:57.205588 796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "7f80ab09958c40c9bdb5bec14342fe87"
format_stamp: "Formatted at 2026-06-19 15:01:57 on dist-test-slave-76rk"
server_key: "afea55361844083e54aadac3c39b2819"
server_key_iv: "5c668c7d44ae98a466802b6f16b0cfd5"
server_key_version: "encryptionkey@0"
I20260619 15:01:57.212359 796 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260619 15:01:57.217211 811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:57.219187 796 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 15:01:57.219391 796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "7f80ab09958c40c9bdb5bec14342fe87"
format_stamp: "Formatted at 2026-06-19 15:01:57 on dist-test-slave-76rk"
server_key: "afea55361844083e54aadac3c39b2819"
server_key_iv: "5c668c7d44ae98a466802b6f16b0cfd5"
server_key_version: "encryptionkey@0"
I20260619 15:01:57.219620 796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:57.226022 796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:57.231036 796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:57.231338 796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:57.252298 796 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40891
I20260619 15:01:57.252399 862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40891 every 8 connection(s)
I20260619 15:01:57.254295 796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:57.257858 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 796
I20260619 15:01:57.258069 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:57.258697 11967 external_mini_cluster.cc:1428] Setting key 85c07f1c326e22147e80f0e9e9b10233
I20260619 15:01:57.259199 863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:57.268644 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:57.260992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:51965 (local address 127.0.0.1:40891)
0619 15:01:57.262017 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:57.262031 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:57.262078 (+ 47us) server_negotiation.cc:407] Connection header received
0619 15:01:57.264252 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:57.264287 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:57.264749 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:57.265380 (+ 631us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:57.266790 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:57.266797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:57.266812 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:57.266831 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:57.266915 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:57.266938 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:57.267205 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:57.267377 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 15:01:57.267603 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":505,"threads_started":1}
I20260619 15:01:57.273168 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:01:57.274726 863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87: Bootstrap starting.
I20260619 15:01:57.278425 863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:57.279958 863 log.cc:824] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:57.280900 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:57.278136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:60567 (local address 127.0.0.1:40891)
0619 15:01:57.278372 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:57.278379 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:57.278696 (+ 317us) server_negotiation.cc:407] Connection header received
0619 15:01:57.278895 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:57.278898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:57.278956 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:57.279048 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:57.280046 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:57.280050 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:57.280053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:01:57.280070 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:57.280155 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:01:57.280187 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:57.280296 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:57.280433 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 15:01:57.280486 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260619 15:01:57.283303 863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87: No bootstrap required, opened a new log
I20260619 15:01:57.290815 863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f80ab09958c40c9bdb5bec14342fe87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260619 15:01:57.291225 863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:57.291302 863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f80ab09958c40c9bdb5bec14342fe87, State: Initialized, Role: FOLLOWER
I20260619 15:01:57.291852 863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f80ab09958c40c9bdb5bec14342fe87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260619 15:01:57.292064 863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:57.292165 863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:57.292287 863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:57.294562 863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f80ab09958c40c9bdb5bec14342fe87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260619 15:01:57.295135 863 leader_election.cc:302] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f80ab09958c40c9bdb5bec14342fe87; no voters:
I20260619 15:01:57.295697 863 leader_election.cc:288] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:57.295879 873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:57.296638 873 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [term 1 LEADER]: Becoming Leader. State: Replica: 7f80ab09958c40c9bdb5bec14342fe87, State: Running, Role: LEADER
I20260619 15:01:57.297253 873 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f80ab09958c40c9bdb5bec14342fe87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260619 15:01:57.298106 863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:57.300544 874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f80ab09958c40c9bdb5bec14342fe87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f80ab09958c40c9bdb5bec14342fe87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } } }
I20260619 15:01:57.300668 875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f80ab09958c40c9bdb5bec14342fe87. Latest consensus state: current_term: 1 leader_uuid: "7f80ab09958c40c9bdb5bec14342fe87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f80ab09958c40c9bdb5bec14342fe87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } } }
I20260619 15:01:57.300995 874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:57.301092 875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:57.303517 878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:57.318182 878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:57.329018 878 catalog_manager.cc:1347] Generated new cluster ID: aafea5f364214288a44e23fc3e8b44ea
I20260619 15:01:57.329150 878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:57.350679 878 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:57.351148 878 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:57.358533 878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f80ab09958c40c9bdb5bec14342fe87: Generated new TSK 0
I20260619 15:01:57.359546 878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:57.369441 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 796
W20260619 15:01:57.370913 858 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.48:51965
W20260619 15:01:57.371724 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 796: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:01:57.371901 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 796
2026-06-19T15:01:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2061 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:57 dist-test-slave-76rk krb5kdc[894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:57 dist-test-slave-76rk krb5kdc[894](info): set up 2 sockets
Jun 19 15:01:57 dist-test-slave-76rk krb5kdc[894](info): commencing operation
krb5kdc: starting...
W20260619 15:01:58.507331 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.006s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:01:58 dist-test-slave-76rk krb5kdc[894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:01:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:01:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:01:58.807514 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34687
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:01:59.191215 907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:01:59.191572 907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:01:59.191751 907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:01:59.200410 907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:01:59.200557 907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:01:59.200683 907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:01:59.200786 907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:01:59.200855 907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:01:59.212540 907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34687
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:01:59.213999 907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:01:59.215490 907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:01:59.229403 912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:59.229485 913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:01:59.229563 915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:01:59.230587 907 server_base.cc:1034] running on GCE node
I20260619 15:01:59.231567 907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:01:59.233263 907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:01:59.234525 907 hybrid_clock.cc:648] HybridClock initialized: now 1781881319234466 us; error 53 us; skew 500 ppm
I20260619 15:01:59.235008 907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:01:59 dist-test-slave-76rk krb5kdc[894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881319, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:01:59.239033 907 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:01:59.241442 907 webserver.cc:460] Webserver started at http://127.0.0.1:44699/ using document root <none> and password file <none>
I20260619 15:01:59.242270 907 fs_manager.cc:359] Metadata directory not provided
I20260619 15:01:59.242396 907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:01:59.242846 907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:01:59.246460 907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "3be7e3e7b4c94605902707b9e9365cc6"
format_stamp: "Formatted at 2026-06-19 15:01:59 on dist-test-slave-76rk"
server_key: "a7f8ab7443c41ebc693fc5cf4a87baf6"
server_key_iv: "620d728a3e032d10f9f2cf5255a345a4"
server_key_version: "encryptionkey@0"
I20260619 15:01:59.247534 907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "3be7e3e7b4c94605902707b9e9365cc6"
format_stamp: "Formatted at 2026-06-19 15:01:59 on dist-test-slave-76rk"
server_key: "a7f8ab7443c41ebc693fc5cf4a87baf6"
server_key_iv: "620d728a3e032d10f9f2cf5255a345a4"
server_key_version: "encryptionkey@0"
I20260619 15:01:59.253882 907 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260619 15:01:59.258517 922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:01:59.260298 907 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 15:01:59.260567 907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "3be7e3e7b4c94605902707b9e9365cc6"
format_stamp: "Formatted at 2026-06-19 15:01:59 on dist-test-slave-76rk"
server_key: "a7f8ab7443c41ebc693fc5cf4a87baf6"
server_key_iv: "620d728a3e032d10f9f2cf5255a345a4"
server_key_version: "encryptionkey@0"
I20260619 15:01:59.260829 907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:01:59.281514 907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:01:59.287360 907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:01:59.287770 907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:01:59.311620 907 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34687
I20260619 15:01:59.311661 973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34687 every 8 connection(s)
I20260619 15:01:59.313270 907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:01:59.318905 974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:01:59.319939 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 907
I20260619 15:01:59.320317 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:01:59.321000 11967 external_mini_cluster.cc:1428] Setting key 8dd2815e69ee34964315efe560ad90dc
I20260619 15:01:59.336678 974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6: Bootstrap starting.
Jun 19 15:01:59 dist-test-slave-76rk krb5kdc[894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 15:01:59.341195 974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6: Neither blocks nor log segments found. Creating new log.
I20260619 15:01:59.343001 974 log.cc:824] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6: Log is configured to *not* fsync() on all Append() calls
I20260619 15:01:59.347329 974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6: No bootstrap required, opened a new log
I20260619 15:01:59.353112 977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:59.323191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:45543 (local address 127.0.0.1:34687)
0619 15:01:59.324475 (+ 1284us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:59.324490 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:59.324538 (+ 48us) server_negotiation.cc:407] Connection header received
0619 15:01:59.327222 (+ 2684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:59.327257 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:59.327854 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:59.328275 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:59.330293 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:59.331885 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:59.334108 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:59.335040 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:59.335211 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:59.340799 (+ 5588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:59.340814 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:59.340840 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:59.340900 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:59.346041 (+ 5141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:59.347247 (+ 1206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:59.347257 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:59.347267 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:59.347611 (+ 344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:59.348386 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:59.348391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:59.348394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:59.349219 (+ 825us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:59.349604 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:59.350499 (+ 895us) server_negotiation.cc:299] Negotiation successful
0619 15:01:59.351411 (+ 912us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1022,"thread_start_us":668,"threads_started":1}
I20260619 15:01:59.358247 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:01:59.358889 974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be7e3e7b4c94605902707b9e9365cc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } }
I20260619 15:01:59.359354 974 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:01:59.359469 974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3be7e3e7b4c94605902707b9e9365cc6, State: Initialized, Role: FOLLOWER
I20260619 15:01:59.360157 974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be7e3e7b4c94605902707b9e9365cc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } }
I20260619 15:01:59.360448 974 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:01:59.360615 974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:01:59.360805 974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:01:59.364367 974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be7e3e7b4c94605902707b9e9365cc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } }
I20260619 15:01:59.365042 974 leader_election.cc:302] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3be7e3e7b4c94605902707b9e9365cc6; no voters:
I20260619 15:01:59.365861 974 leader_election.cc:288] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:01:59.366010 984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:01:59.367477 984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [term 1 LEADER]: Becoming Leader. State: Replica: 3be7e3e7b4c94605902707b9e9365cc6, State: Running, Role: LEADER
I20260619 15:01:59.368417 984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be7e3e7b4c94605902707b9e9365cc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } }
I20260619 15:01:59.368983 974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:01:59.372197 985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3be7e3e7b4c94605902707b9e9365cc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be7e3e7b4c94605902707b9e9365cc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } } }
I20260619 15:01:59.372583 986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3be7e3e7b4c94605902707b9e9365cc6. Latest consensus state: current_term: 1 leader_uuid: "3be7e3e7b4c94605902707b9e9365cc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be7e3e7b4c94605902707b9e9365cc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } } }
I20260619 15:01:59.372999 985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:59.373006 986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6 [sys.catalog]: This master's current role is: LEADER
I20260619 15:01:59.374184 992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:01:59.388350 977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:01:59.363590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:42279 (local address 127.0.0.1:34687)
0619 15:01:59.363866 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0619 15:01:59.363876 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:01:59.364381 (+ 505us) server_negotiation.cc:407] Connection header received
0619 15:01:59.364689 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:01:59.364693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:01:59.364766 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:01:59.364886 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:01:59.367178 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:59.368397 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:59.369934 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:01:59.370837 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:01:59.371002 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:01:59.375792 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:01:59.375800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:01:59.375804 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:01:59.375853 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:01:59.381080 (+ 5227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:59.382895 (+ 1815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:59.382906 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:59.382912 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:59.383213 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:01:59.384282 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:01:59.384291 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:01:59.384297 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:01:59.384916 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:01:59.385292 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0619 15:01:59.387310 (+ 2018us) server_negotiation.cc:299] Negotiation successful
0619 15:01:59.387629 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 15:01:59.395537 992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:01:59.407740 992 catalog_manager.cc:1347] Generated new cluster ID: f1bda56eac9c4316be92b62293127608
I20260619 15:01:59.407893 992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:01:59.437703 992 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:01:59.439810 992 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:01:59.460250 992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3be7e3e7b4c94605902707b9e9365cc6: Generated new TSK 0
I20260619 15:01:59.461349 992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:01:59.480830 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 907
I20260619 15:01:59.482143 969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:01:59.582353 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 907
W20260619 15:01:59.601748 892 connection.cc:535] client connection to 127.0.0.1:34687 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34687): Connection reset by peer (error 104)
2026-06-19T15:01:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2215 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:01:59 dist-test-slave-76rk krb5kdc[1006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:01:59 dist-test-slave-76rk krb5kdc[1006](info): set up 2 sockets
Jun 19 15:01:59 dist-test-slave-76rk krb5kdc[1006](info): commencing operation
krb5kdc: starting...
W20260619 15:02:00.747200 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:02:00 dist-test-slave-76rk krb5kdc[1006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:02:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:02:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:02:01.043514 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:02:01.399753 1019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:01.400069 1019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:01.400243 1019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:01.408129 1019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:01.408274 1019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:01.408380 1019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:01.408479 1019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:01.408591 1019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:01.419943 1019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:01.421458 1019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:01.422940 1019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:01.435940 1024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:01.435986 1025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:01.436143 1027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:01.437212 1019 server_base.cc:1034] running on GCE node
I20260619 15:02:01.438120 1019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:01.439742 1019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:01.440968 1019 hybrid_clock.cc:648] HybridClock initialized: now 1781881321440915 us; error 41 us; skew 500 ppm
I20260619 15:02:01.441388 1019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:01 dist-test-slave-76rk krb5kdc[1006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:01.444802 1019 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:02:01.446591 1019 webserver.cc:460] Webserver started at http://127.0.0.1:43681/ using document root <none> and password file <none>
I20260619 15:02:01.447402 1019 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:01.447557 1019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:01.447886 1019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:01.451165 1019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "d5e0cb92b701487a99ca779a7a5cd97a"
format_stamp: "Formatted at 2026-06-19 15:02:01 on dist-test-slave-76rk"
server_key: "5bedff6d6968dbd5c15a718beae46f8e"
server_key_iv: "02fa741ae5f42514961110e441f05139"
server_key_version: "encryptionkey@0"
I20260619 15:02:01.452067 1019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "d5e0cb92b701487a99ca779a7a5cd97a"
format_stamp: "Formatted at 2026-06-19 15:02:01 on dist-test-slave-76rk"
server_key: "5bedff6d6968dbd5c15a718beae46f8e"
server_key_iv: "02fa741ae5f42514961110e441f05139"
server_key_version: "encryptionkey@0"
I20260619 15:02:01.457895 1019 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 15:02:01.462090 1034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:01.463805 1019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 15:02:01.464051 1019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "d5e0cb92b701487a99ca779a7a5cd97a"
format_stamp: "Formatted at 2026-06-19 15:02:01 on dist-test-slave-76rk"
server_key: "5bedff6d6968dbd5c15a718beae46f8e"
server_key_iv: "02fa741ae5f42514961110e441f05139"
server_key_version: "encryptionkey@0"
I20260619 15:02:01.464308 1019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:01.479665 1019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:01.484846 1019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:01.485158 1019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:01.504397 1019 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46215
I20260619 15:02:01.504498 1085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46215 every 8 connection(s)
I20260619 15:02:01.505924 1019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:02:01.510185 1086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:01.513051 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 1019
I20260619 15:02:01.513260 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:02:01.513789 11967 external_mini_cluster.cc:1428] Setting key 71c7d5474342f1ffeb705ba1c0ce45a4
I20260619 15:02:01.524291 1086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a: Bootstrap starting.
I20260619 15:02:01.529505 1086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:01.530571 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:01.515997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:54387 (local address 127.0.0.1:46215)
0619 15:02:01.516902 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:01.516915 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:01.516960 (+ 45us) server_negotiation.cc:407] Connection header received
0619 15:02:01.518832 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:01.518859 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:01.519314 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:01.519671 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:01.520364 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:01.521849 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:01.524344 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:01.525228 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:01.525394 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:01.527817 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:01.527826 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:01.527844 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:01.528455 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:01.528548 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:01.528579 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:01.528726 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:01.529135 (+ 409us) server_negotiation.cc:299] Negotiation successful
0619 15:02:01.529356 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":413,"threads_started":1}
I20260619 15:02:01.531111 1086 log.cc:824] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:01.534560 1086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a: No bootstrap required, opened a new log
I20260619 15:02:01.535952 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:02:01.545817 1086 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e0cb92b701487a99ca779a7a5cd97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } }
I20260619 15:02:01.546243 1086 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:02:01.546403 1086 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5e0cb92b701487a99ca779a7a5cd97a, State: Initialized, Role: FOLLOWER
I20260619 15:02:01.547237 1086 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e0cb92b701487a99ca779a7a5cd97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } }
I20260619 15:02:01.547590 1086 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:02:01.547783 1086 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:02:01.548012 1086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:01.549873 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:01.541862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:52339 (local address 127.0.0.1:46215)
0619 15:02:01.542081 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:01.542090 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:01.542725 (+ 635us) server_negotiation.cc:407] Connection header received
0619 15:02:01.542895 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:01.542901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:01.542986 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:01.543110 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:01.543867 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:01.545051 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:01.546770 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:01.547462 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:01.547609 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:01.549005 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:01.549013 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:01.549018 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:01.549042 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:01.549097 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:01.549119 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:01.549244 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:01.549455 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 15:02:01.549518 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 15:02:01.551400 1086 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e0cb92b701487a99ca779a7a5cd97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } }
I20260619 15:02:01.551985 1086 leader_election.cc:302] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5e0cb92b701487a99ca779a7a5cd97a; no voters:
I20260619 15:02:01.552636 1086 leader_election.cc:288] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:02:01.552819 1096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:02:01.553608 1096 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [term 1 LEADER]: Becoming Leader. State: Replica: d5e0cb92b701487a99ca779a7a5cd97a, State: Running, Role: LEADER
I20260619 15:02:01.554235 1096 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e0cb92b701487a99ca779a7a5cd97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } }
I20260619 15:02:01.555060 1086 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:01.557334 1097 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5e0cb92b701487a99ca779a7a5cd97a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e0cb92b701487a99ca779a7a5cd97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } } }
I20260619 15:02:01.557416 1098 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5e0cb92b701487a99ca779a7a5cd97a. Latest consensus state: current_term: 1 leader_uuid: "d5e0cb92b701487a99ca779a7a5cd97a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e0cb92b701487a99ca779a7a5cd97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } } }
I20260619 15:02:01.558084 1097 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:01.558351 1098 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:01.567447 1105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:02:01.572379 1105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:02:01.581658 1105 catalog_manager.cc:1347] Generated new cluster ID: 29d886978af442cba1e83cfe5d912ea6
I20260619 15:02:01.581815 1105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:02:01.594410 1105 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:02:01.596575 1105 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:02:01.610788 1105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5e0cb92b701487a99ca779a7a5cd97a: Generated new TSK 0
I20260619 15:02:01.611855 1105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:02:01.624965 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1019
W20260619 15:02:01.627089 1081 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.48:54387
W20260619 15:02:01.627931 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1019: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:02:01.628088 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 1019
W20260619 15:02:01.651573 1004 connection.cc:535] client connection to 127.0.0.1:46215 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46215): Connection reset by peer (error 104)
2026-06-19T15:02:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2050 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:02:01 dist-test-slave-76rk krb5kdc[1117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:02:01 dist-test-slave-76rk krb5kdc[1117](info): set up 2 sockets
Jun 19 15:02:01 dist-test-slave-76rk krb5kdc[1117](info): commencing operation
krb5kdc: starting...
W20260619 15:02:02.791518 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.004s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:02:02 dist-test-slave-76rk krb5kdc[1117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:02:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:02:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 15:02:03.084096 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:02:03.446118 1130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:03.446455 1130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:03.446624 1130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:03.454255 1130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:03.454385 1130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:03.454484 1130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:03.454591 1130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:03.454700 1130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:03.465461 1130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42671
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:03.466924 1130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:03.468348 1130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:03.481639 1138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:03.481614 1136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:03.481978 1135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:03.482551 1130 server_base.cc:1034] running on GCE node
I20260619 15:02:03.483439 1130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:03.484901 1130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:03.486129 1130 hybrid_clock.cc:648] HybridClock initialized: now 1781881323486086 us; error 38 us; skew 500 ppm
I20260619 15:02:03.486532 1130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:03 dist-test-slave-76rk krb5kdc[1117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881323, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:03.490213 1130 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 15:02:03.491900 1130 webserver.cc:460] Webserver started at http://127.0.0.1:36737/ using document root <none> and password file <none>
I20260619 15:02:03.492588 1130 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:03.492697 1130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:03.492966 1130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:03.496176 1130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "6dfab96df0d44d88b4e224d2d62d29a7"
format_stamp: "Formatted at 2026-06-19 15:02:03 on dist-test-slave-76rk"
server_key: "5d024f54a6bd7d320916f77680ec162c"
server_key_iv: "3c011971a3c6afa9178ce99588b3daa0"
server_key_version: "encryptionkey@0"
I20260619 15:02:03.496932 1130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "6dfab96df0d44d88b4e224d2d62d29a7"
format_stamp: "Formatted at 2026-06-19 15:02:03 on dist-test-slave-76rk"
server_key: "5d024f54a6bd7d320916f77680ec162c"
server_key_iv: "3c011971a3c6afa9178ce99588b3daa0"
server_key_version: "encryptionkey@0"
I20260619 15:02:03.502522 1130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 15:02:03.506966 1145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:03.508663 1130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260619 15:02:03.508869 1130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "6dfab96df0d44d88b4e224d2d62d29a7"
format_stamp: "Formatted at 2026-06-19 15:02:03 on dist-test-slave-76rk"
server_key: "5d024f54a6bd7d320916f77680ec162c"
server_key_iv: "3c011971a3c6afa9178ce99588b3daa0"
server_key_version: "encryptionkey@0"
I20260619 15:02:03.509083 1130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:03.515538 1130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:03.520251 1130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:03.520567 1130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:03.540894 1130 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42671
I20260619 15:02:03.540927 1196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42671 every 8 connection(s)
I20260619 15:02:03.542464 1130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:02:03.544355 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 1130
I20260619 15:02:03.544562 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:02:03.545035 11967 external_mini_cluster.cc:1428] Setting key 7728657e8c975718233cdd5caac63c06
I20260619 15:02:03.547171 1197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:03.553341 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:03.546880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:38411 (local address 127.0.0.1:42671)
0619 15:02:03.547713 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:03.547725 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:03.547768 (+ 43us) server_negotiation.cc:407] Connection header received
0619 15:02:03.549418 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:03.549441 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:03.549811 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:03.550165 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:03.551407 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:03.551414 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:03.551427 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:03.551443 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:03.551529 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:03.551551 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:03.551742 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:03.551965 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 15:02:03.552262 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":381,"threads_started":1}
I20260619 15:02:03.557442 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:02:03.560295 1197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7: Bootstrap starting.
I20260619 15:02:03.564663 1197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:03.565588 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:03.562931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:47073 (local address 127.0.0.1:42671)
0619 15:02:03.563138 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:03.563144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:03.563426 (+ 282us) server_negotiation.cc:407] Connection header received
0619 15:02:03.563594 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:03.563599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:03.563657 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:03.563746 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:03.564824 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:03.564831 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:03.564838 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:03.564862 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:03.564906 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:03.564923 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:03.565018 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:03.565210 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 15:02:03.565275 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 15:02:03.566089 1197 log.cc:824] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:03.569295 1197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7: No bootstrap required, opened a new log
I20260619 15:02:03.577762 1197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfab96df0d44d88b4e224d2d62d29a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } }
I20260619 15:02:03.578166 1197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:02:03.578251 1197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dfab96df0d44d88b4e224d2d62d29a7, State: Initialized, Role: FOLLOWER
I20260619 15:02:03.578891 1197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfab96df0d44d88b4e224d2d62d29a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } }
I20260619 15:02:03.579101 1197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:02:03.579326 1197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:02:03.579490 1197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:03.582095 1197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfab96df0d44d88b4e224d2d62d29a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } }
I20260619 15:02:03.582581 1197 leader_election.cc:302] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dfab96df0d44d88b4e224d2d62d29a7; no voters:
I20260619 15:02:03.583209 1197 leader_election.cc:288] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:02:03.583361 1207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:02:03.584060 1207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [term 1 LEADER]: Becoming Leader. State: Replica: 6dfab96df0d44d88b4e224d2d62d29a7, State: Running, Role: LEADER
I20260619 15:02:03.584643 1207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfab96df0d44d88b4e224d2d62d29a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } }
I20260619 15:02:03.585402 1197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:03.587997 1208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dfab96df0d44d88b4e224d2d62d29a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfab96df0d44d88b4e224d2d62d29a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } } }
I20260619 15:02:03.588661 1208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:03.588332 1209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dfab96df0d44d88b4e224d2d62d29a7. Latest consensus state: current_term: 1 leader_uuid: "6dfab96df0d44d88b4e224d2d62d29a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfab96df0d44d88b4e224d2d62d29a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42671 } } }
I20260619 15:02:03.588811 1209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:03.591974 1216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:02:03.597764 1216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:02:03.606189 1216 catalog_manager.cc:1347] Generated new cluster ID: e5d1e664e19c4b83974a6873a2cb9347
I20260619 15:02:03.606389 1216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:02:03.631968 1216 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:02:03.632484 1216 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:02:03.653546 1216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dfab96df0d44d88b4e224d2d62d29a7: Generated new TSK 0
I20260619 15:02:03.654495 1216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:02:03.672262 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1130
W20260619 15:02:03.673871 1192 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.48:38411
W20260619 15:02:03.674746 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1130: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:02:03.674912 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 1130
2026-06-19T15:02:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2040 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:02:03 dist-test-slave-76rk krb5kdc[1228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:02:03 dist-test-slave-76rk krb5kdc[1228](info): set up 2 sockets
Jun 19 15:02:03 dist-test-slave-76rk krb5kdc[1228](info): commencing operation
krb5kdc: starting...
W20260619 15:02:04.803421 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:02:04 dist-test-slave-76rk krb5kdc[1228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:02:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:02:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:02:05.098260 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:34355
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:41455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:34355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:02:05.447223 1241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:05.447530 1241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:05.447623 1241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:05.455055 1241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:05.455183 1241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:05.455230 1241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:05.455291 1241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:05.455336 1241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:05.465443 1241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:41455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:34355
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:34355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:05.466802 1241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:05.468066 1241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:05.480134 1249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:05.480193 1246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:05.480110 1247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:05.480880 1241 server_base.cc:1034] running on GCE node
I20260619 15:02:05.481648 1241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:05.483235 1241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:05.484453 1241 hybrid_clock.cc:648] HybridClock initialized: now 1781881325484413 us; error 29 us; skew 500 ppm
I20260619 15:02:05.484839 1241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:05 dist-test-slave-76rk krb5kdc[1228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881325, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:05.488025 1241 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:02:05.489749 1241 webserver.cc:460] Webserver started at http://127.11.175.254:41901/ using document root <none> and password file <none>
I20260619 15:02:05.490772 1241 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:05.490917 1241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:05.491250 1241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:05.494328 1241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "9a568c8e10d448bab5f00b1ce07b8544"
format_stamp: "Formatted at 2026-06-19 15:02:05 on dist-test-slave-76rk"
server_key: "597073803aa963ae44d110603d5172ee"
server_key_iv: "87a20ace16dee41ba13f4e45a1096b2e"
server_key_version: "encryptionkey@0"
I20260619 15:02:05.495199 1241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "9a568c8e10d448bab5f00b1ce07b8544"
format_stamp: "Formatted at 2026-06-19 15:02:05 on dist-test-slave-76rk"
server_key: "597073803aa963ae44d110603d5172ee"
server_key_iv: "87a20ace16dee41ba13f4e45a1096b2e"
server_key_version: "encryptionkey@0"
I20260619 15:02:05.501055 1241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 15:02:05.504839 1256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:05.506321 1241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 15:02:05.506516 1241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "9a568c8e10d448bab5f00b1ce07b8544"
format_stamp: "Formatted at 2026-06-19 15:02:05 on dist-test-slave-76rk"
server_key: "597073803aa963ae44d110603d5172ee"
server_key_iv: "87a20ace16dee41ba13f4e45a1096b2e"
server_key_version: "encryptionkey@0"
I20260619 15:02:05.506796 1241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:05.526822 1241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:05.531605 1241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:05.531925 1241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:05.551249 1241 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:34355
I20260619 15:02:05.551286 1307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:34355 every 8 connection(s)
I20260619 15:02:05.552722 1241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:02:05.556361 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 1241
I20260619 15:02:05.556612 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:02:05.556999 1308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:05.557139 11967 external_mini_cluster.cc:1428] Setting key 735a59aa108349846efb3a4a177b58c4
I20260619 15:02:05.572124 1308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544: Bootstrap starting.
Jun 19 15:02:05 dist-test-slave-76rk krb5kdc[1228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:02:05.576149 1308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:05.577356 1308 log.cc:824] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:05.580331 1308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544: No bootstrap required, opened a new log
I20260619 15:02:05.582608 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:05.559118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35347 (local address 127.11.175.254:34355)
0619 15:02:05.559932 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:05.559949 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:05.560004 (+ 55us) server_negotiation.cc:407] Connection header received
0619 15:02:05.562203 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:05.562233 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:05.562845 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:05.563332 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:05.564899 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:05.566954 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:05.568301 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:05.569459 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:05.569679 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:05.574092 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:05.574104 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:05.574124 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:05.574185 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:05.578005 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:05.579034 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:05.579042 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:05.579048 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:05.579223 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:05.579747 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:05.579750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:05.579752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:05.580273 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:05.580627 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:05.581187 (+ 560us) server_negotiation.cc:299] Negotiation successful
0619 15:02:05.581523 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":329,"threads_started":1}
I20260619 15:02:05.586807 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:02:05.588822 1308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a568c8e10d448bab5f00b1ce07b8544" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34355 } }
I20260619 15:02:05.589248 1308 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:02:05.589380 1308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a568c8e10d448bab5f00b1ce07b8544, State: Initialized, Role: FOLLOWER
I20260619 15:02:05.590251 1308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a568c8e10d448bab5f00b1ce07b8544" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34355 } }
I20260619 15:02:05.590592 1308 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:02:05.590780 1308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:02:05.591001 1308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:05.593799 1308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a568c8e10d448bab5f00b1ce07b8544" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34355 } }
I20260619 15:02:05.594406 1308 leader_election.cc:302] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a568c8e10d448bab5f00b1ce07b8544; no voters:
I20260619 15:02:05.595091 1308 leader_election.cc:288] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:02:05.595222 1318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:02:05.596379 1318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [term 1 LEADER]: Becoming Leader. State: Replica: 9a568c8e10d448bab5f00b1ce07b8544, State: Running, Role: LEADER
I20260619 15:02:05.597036 1318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a568c8e10d448bab5f00b1ce07b8544" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34355 } }
I20260619 15:02:05.597570 1308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:05.603039 1319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a568c8e10d448bab5f00b1ce07b8544" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a568c8e10d448bab5f00b1ce07b8544" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34355 } } }
I20260619 15:02:05.604238 1319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:05.603878 1320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a568c8e10d448bab5f00b1ce07b8544. Latest consensus state: current_term: 1 leader_uuid: "9a568c8e10d448bab5f00b1ce07b8544" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a568c8e10d448bab5f00b1ce07b8544" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 34355 } } }
I20260619 15:02:05.604980 1320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:05.607434 1327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:02:05.612759 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:05.591824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56119 (local address 127.11.175.254:34355)
0619 15:02:05.592040 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:05.592049 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:05.592457 (+ 408us) server_negotiation.cc:407] Connection header received
0619 15:02:05.592673 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:05.592678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:05.592754 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:05.592873 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:05.594380 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:05.595582 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:05.597634 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:05.598316 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:05.598469 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:05.602463 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:05.602470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:05.602475 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:05.602523 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:05.606943 (+ 4420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:05.608481 (+ 1538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:05.608490 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:05.608496 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:05.608677 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:05.609408 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:05.609417 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:05.609423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:05.610123 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:05.610506 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:05.611965 (+ 1459us) server_negotiation.cc:299] Negotiation successful
0619 15:02:05.612223 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 15:02:05.615013 1327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:02:05.630492 1327 catalog_manager.cc:1347] Generated new cluster ID: 04ebee35d8f54912994905670fb24c3b
I20260619 15:02:05.630662 1327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:02:05.642086 1327 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:02:05.644153 1327 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:02:05.664404 1327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a568c8e10d448bab5f00b1ce07b8544: Generated new TSK 0
I20260619 15:02:05.665405 1327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:02:05.680485 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1241
I20260619 15:02:05.681908 1303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:02:05.776881 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 1241
W20260619 15:02:05.794533 1226 connection.cc:535] client connection to 127.11.175.254:34355 recv error: Network error: failed to read from TLS socket (remote: 127.11.175.254:34355): Connection reset by peer (error 104)
2026-06-19T15:02:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2101 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:02:05 dist-test-slave-76rk krb5kdc[1341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:02:05 dist-test-slave-76rk krb5kdc[1341](info): set up 2 sockets
Jun 19 15:02:05 dist-test-slave-76rk krb5kdc[1341](info): commencing operation
krb5kdc: starting...
W20260619 15:02:06.907292 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.006s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:02:07 dist-test-slave-76rk krb5kdc[1341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:02:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:02:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:02:07.193719 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:37097
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:38315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:37097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:02:07.536609 1354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:07.536954 1354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:07.537129 1354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:07.544755 1354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:07.544890 1354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:07.544984 1354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:07.545099 1354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:07.545168 1354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:07.555562 1354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:38315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:37097
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:37097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:07.557010 1354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:07.558414 1354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:07.570920 1359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:07.571252 1360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:07.571481 1362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:07.572342 1354 server_base.cc:1034] running on GCE node
I20260619 15:02:07.573426 1354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:07.575127 1354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:07.576488 1354 hybrid_clock.cc:648] HybridClock initialized: now 1781881327576413 us; error 67 us; skew 500 ppm
I20260619 15:02:07.576983 1354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:07 dist-test-slave-76rk krb5kdc[1341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881327, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:07.592025 1354 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:02:07.594081 1354 webserver.cc:460] Webserver started at http://127.11.175.254:41061/ using document root <none> and password file <none>
I20260619 15:02:07.595297 1354 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:07.595503 1354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:07.595913 1354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:07.599732 1354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "fe4c8aaf689f4049888909b7383d20e3"
format_stamp: "Formatted at 2026-06-19 15:02:07 on dist-test-slave-76rk"
server_key: "4874727a253acebfa7169150ceb2ccc0"
server_key_iv: "fc8911fd01d747128395d9df87ace79f"
server_key_version: "encryptionkey@0"
I20260619 15:02:07.600759 1354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "fe4c8aaf689f4049888909b7383d20e3"
format_stamp: "Formatted at 2026-06-19 15:02:07 on dist-test-slave-76rk"
server_key: "4874727a253acebfa7169150ceb2ccc0"
server_key_iv: "fc8911fd01d747128395d9df87ace79f"
server_key_version: "encryptionkey@0"
I20260619 15:02:07.606875 1354 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 15:02:07.610904 1369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:07.612430 1354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 15:02:07.612654 1354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "fe4c8aaf689f4049888909b7383d20e3"
format_stamp: "Formatted at 2026-06-19 15:02:07 on dist-test-slave-76rk"
server_key: "4874727a253acebfa7169150ceb2ccc0"
server_key_iv: "fc8911fd01d747128395d9df87ace79f"
server_key_version: "encryptionkey@0"
I20260619 15:02:07.612926 1354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:07.627897 1354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:07.632627 1354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:07.632941 1354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:07.654522 1354 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:37097
I20260619 15:02:07.654552 1420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:37097 every 8 connection(s)
I20260619 15:02:07.656210 1354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:02:07.660184 1421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:07.662546 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 1354
I20260619 15:02:07.662871 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:02:07.663319 11967 external_mini_cluster.cc:1428] Setting key 625e58500f10e4958d3cbb7ae498e6ea
I20260619 15:02:07.672946 1421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3: Bootstrap starting.
I20260619 15:02:07.676611 1421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:07.677786 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:07.665229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39517 (local address 127.11.175.254:37097)
0619 15:02:07.666089 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:07.666104 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:07.666147 (+ 43us) server_negotiation.cc:407] Connection header received
0619 15:02:07.668040 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:07.668066 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:07.668525 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:07.669033 (+ 508us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:07.669701 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:07.671353 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:07.672807 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:07.673591 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:07.673803 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:07.675264 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:07.675273 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:07.675289 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:07.675871 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:07.675954 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:07.675983 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:07.676161 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:07.676483 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 15:02:07.676678 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":380,"threads_started":1}
I20260619 15:02:07.677831 1421 log.cc:824] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:07.681195 1421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3: No bootstrap required, opened a new log
I20260619 15:02:07.682380 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:02:07.691120 1421 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4c8aaf689f4049888909b7383d20e3" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37097 } }
I20260619 15:02:07.691697 1421 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:02:07.691820 1421 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe4c8aaf689f4049888909b7383d20e3, State: Initialized, Role: FOLLOWER
I20260619 15:02:07.692515 1421 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4c8aaf689f4049888909b7383d20e3" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37097 } }
I20260619 15:02:07.692809 1421 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:02:07.693008 1421 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:02:07.693221 1421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:07.695118 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:07.686949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43019 (local address 127.11.175.254:37097)
0619 15:02:07.687200 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:07.687210 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:07.687756 (+ 546us) server_negotiation.cc:407] Connection header received
0619 15:02:07.687982 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:07.687987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:07.688059 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:07.688215 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:07.688970 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:07.689840 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:07.692077 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:07.692706 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:07.692882 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:07.694218 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:07.694223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:07.694227 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:07.694246 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:07.694343 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:07.694363 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:07.694508 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:07.694746 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 15:02:07.694809 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 15:02:07.696488 1421 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4c8aaf689f4049888909b7383d20e3" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37097 } }
I20260619 15:02:07.697161 1421 leader_election.cc:302] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe4c8aaf689f4049888909b7383d20e3; no voters:
I20260619 15:02:07.697721 1421 leader_election.cc:288] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:02:07.697891 1431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:02:07.698730 1431 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [term 1 LEADER]: Becoming Leader. State: Replica: fe4c8aaf689f4049888909b7383d20e3, State: Running, Role: LEADER
I20260619 15:02:07.699332 1431 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4c8aaf689f4049888909b7383d20e3" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37097 } }
I20260619 15:02:07.700038 1421 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:07.702603 1433 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe4c8aaf689f4049888909b7383d20e3. Latest consensus state: current_term: 1 leader_uuid: "fe4c8aaf689f4049888909b7383d20e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4c8aaf689f4049888909b7383d20e3" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37097 } } }
I20260619 15:02:07.703074 1433 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:07.702574 1432 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe4c8aaf689f4049888909b7383d20e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4c8aaf689f4049888909b7383d20e3" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 37097 } } }
I20260619 15:02:07.703859 1432 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:07.705776 1437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:02:07.711871 1437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:02:07.721743 1437 catalog_manager.cc:1347] Generated new cluster ID: 6c40b9c0e704483596da727a8f518e2b
I20260619 15:02:07.721895 1437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:02:07.758424 1437 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:02:07.759824 1437 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:02:07.768244 1437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe4c8aaf689f4049888909b7383d20e3: Generated new TSK 0
I20260619 15:02:07.769241 1437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:02:07.783965 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1354
W20260619 15:02:07.786240 1416 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39517
W20260619 15:02:07.787176 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1354: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:02:07.787339 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 1354
W20260619 15:02:07.804858 1339 connection.cc:535] client connection to 127.11.175.254:37097 recv error: Network error: failed to read from TLS socket (remote: 127.11.175.254:37097): Connection reset by peer (error 104)
2026-06-19T15:02:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2009 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:02:07 dist-test-slave-76rk krb5kdc[1452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:02:07 dist-test-slave-76rk krb5kdc[1452](info): set up 2 sockets
Jun 19 15:02:07 dist-test-slave-76rk krb5kdc[1452](info): commencing operation
krb5kdc: starting...
W20260619 15:02:08.899101 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:02:09 dist-test-slave-76rk krb5kdc[1452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:02:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:02:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:02:09.182502 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:36605
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:36605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:02:09.527799 1465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:09.528193 1465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:09.528386 1465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:09.536051 1465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:09.536173 1465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:09.536259 1465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:09.536351 1465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:09.536435 1465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:09.546608 1465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:36605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:36605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:09.548136 1465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:09.549472 1465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:09.561120 1470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:09.561168 1471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:09.561333 1473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:09.562410 1465 server_base.cc:1034] running on GCE node
I20260619 15:02:09.563316 1465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:09.564777 1465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:09.566031 1465 hybrid_clock.cc:648] HybridClock initialized: now 1781881329565984 us; error 40 us; skew 500 ppm
I20260619 15:02:09.566471 1465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:09 dist-test-slave-76rk krb5kdc[1452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:09.570118 1465 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:02:09.571842 1465 webserver.cc:460] Webserver started at http://127.11.175.254:46639/ using document root <none> and password file <none>
I20260619 15:02:09.572589 1465 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:09.572742 1465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:09.573069 1465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:09.576372 1465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6"
format_stamp: "Formatted at 2026-06-19 15:02:09 on dist-test-slave-76rk"
server_key: "057a458c70c95f8be7872a404679fe4b"
server_key_iv: "8e6d0e26be50c4e02b7e6c29833ef555"
server_key_version: "encryptionkey@0"
I20260619 15:02:09.577193 1465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6"
format_stamp: "Formatted at 2026-06-19 15:02:09 on dist-test-slave-76rk"
server_key: "057a458c70c95f8be7872a404679fe4b"
server_key_iv: "8e6d0e26be50c4e02b7e6c29833ef555"
server_key_version: "encryptionkey@0"
I20260619 15:02:09.582759 1465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 15:02:09.586450 1480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:09.588106 1465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 15:02:09.588322 1465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6"
format_stamp: "Formatted at 2026-06-19 15:02:09 on dist-test-slave-76rk"
server_key: "057a458c70c95f8be7872a404679fe4b"
server_key_iv: "8e6d0e26be50c4e02b7e6c29833ef555"
server_key_version: "encryptionkey@0"
I20260619 15:02:09.588557 1465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:09.594794 1465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:09.599352 1465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:09.599661 1465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:09.618976 1465 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:36605
I20260619 15:02:09.619117 1531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:36605 every 8 connection(s)
I20260619 15:02:09.620409 1465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:02:09.620709 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 1465
I20260619 15:02:09.620905 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:02:09.621515 11967 external_mini_cluster.cc:1428] Setting key 2f506fa65ae375a1cdad006a6c53d461
I20260619 15:02:09.624913 1532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:09.629920 1535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:09.623473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39149 (local address 127.11.175.254:36605)
0619 15:02:09.624401 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:09.624415 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:09.624465 (+ 50us) server_negotiation.cc:407] Connection header received
0619 15:02:09.626266 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:09.626289 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:09.626619 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:09.627085 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:09.628205 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:09.628211 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:09.628229 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:09.628244 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:09.628320 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:09.628347 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:09.628620 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:09.628786 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 15:02:09.628973 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":360,"threads_started":1}
I20260619 15:02:09.634073 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:02:09.636932 1532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6: Bootstrap starting.
I20260619 15:02:09.640810 1532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:09.641279 1535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:09.638695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45001 (local address 127.11.175.254:36605)
0619 15:02:09.638853 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:09.638863 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:09.639208 (+ 345us) server_negotiation.cc:407] Connection header received
0619 15:02:09.639393 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:09.639397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:09.639446 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:09.639529 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:09.640519 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:09.640524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:09.640527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:09.640541 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:09.640583 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:09.640622 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:09.640717 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:09.640869 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 15:02:09.640958 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 15:02:09.642242 1532 log.cc:824] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:09.645850 1532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6: No bootstrap required, opened a new log
I20260619 15:02:09.654228 1532 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36605 } }
I20260619 15:02:09.654691 1532 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:02:09.654841 1532 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7bfa5c291741cd90c2a0c2d24e52b6, State: Initialized, Role: FOLLOWER
I20260619 15:02:09.655539 1532 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36605 } }
I20260619 15:02:09.655810 1532 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:02:09.655969 1532 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:02:09.656209 1532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:09.658603 1532 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36605 } }
I20260619 15:02:09.659273 1532 leader_election.cc:302] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd7bfa5c291741cd90c2a0c2d24e52b6; no voters:
I20260619 15:02:09.659895 1532 leader_election.cc:288] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:02:09.660071 1542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:02:09.660835 1542 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [term 1 LEADER]: Becoming Leader. State: Replica: cd7bfa5c291741cd90c2a0c2d24e52b6, State: Running, Role: LEADER
I20260619 15:02:09.661412 1542 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36605 } }
I20260619 15:02:09.662504 1532 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:09.665283 1544 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd7bfa5c291741cd90c2a0c2d24e52b6. Latest consensus state: current_term: 1 leader_uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36605 } } }
I20260619 15:02:09.665827 1544 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:09.665256 1543 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7bfa5c291741cd90c2a0c2d24e52b6" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36605 } } }
I20260619 15:02:09.666075 1543 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:09.667934 1549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:02:09.673475 1549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:02:09.684487 1549 catalog_manager.cc:1347] Generated new cluster ID: b08f57904b4b480db8cca69b079f3b87
I20260619 15:02:09.684625 1549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:02:09.698664 1549 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:02:09.699196 1549 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:02:09.711642 1549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd7bfa5c291741cd90c2a0c2d24e52b6: Generated new TSK 0
I20260619 15:02:09.712600 1549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:02:09.720707 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1465
W20260619 15:02:09.721895 1527 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39149
W20260619 15:02:09.722597 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1465: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:02:09.722795 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 1465
2026-06-19T15:02:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1934 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:02:09 dist-test-slave-76rk krb5kdc[1563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:02:09 dist-test-slave-76rk krb5kdc[1563](info): set up 2 sockets
Jun 19 15:02:09 dist-test-slave-76rk krb5kdc[1563](info): commencing operation
krb5kdc: starting...
W20260619 15:02:10.839100 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.006s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:02:11 dist-test-slave-76rk krb5kdc[1563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:02:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:02:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:02:11.117863 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38281
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:41887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:38281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:02:11.455943 1576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:11.456316 1576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:11.456477 1576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:11.463935 1576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:11.464104 1576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:11.464221 1576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:11.464339 1576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:11.464418 1576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:11.474315 1576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:41887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:38281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:38281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:11.475689 1576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:11.477046 1576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:11.488999 1581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:11.489542 1582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:11.489938 1584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:11.490712 1576 server_base.cc:1034] running on GCE node
I20260619 15:02:11.491562 1576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:11.492962 1576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:11.494169 1576 hybrid_clock.cc:648] HybridClock initialized: now 1781881331494099 us; error 57 us; skew 500 ppm
I20260619 15:02:11.494529 1576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:11 dist-test-slave-76rk krb5kdc[1563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881331, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:11.497704 1576 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:02:11.499228 1576 webserver.cc:460] Webserver started at http://127.11.175.254:40885/ using document root <none> and password file <none>
I20260619 15:02:11.499889 1576 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:11.499991 1576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:11.500240 1576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:11.503302 1576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "fd4ba60ca4824e35a1a514752bfba20e"
format_stamp: "Formatted at 2026-06-19 15:02:11 on dist-test-slave-76rk"
server_key: "930bdf795ac563fd2ca9c8f6fdb7c7a9"
server_key_iv: "eed45e7fcb16f3d10ad472ba39067ded"
server_key_version: "encryptionkey@0"
I20260619 15:02:11.504124 1576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "fd4ba60ca4824e35a1a514752bfba20e"
format_stamp: "Formatted at 2026-06-19 15:02:11 on dist-test-slave-76rk"
server_key: "930bdf795ac563fd2ca9c8f6fdb7c7a9"
server_key_iv: "eed45e7fcb16f3d10ad472ba39067ded"
server_key_version: "encryptionkey@0"
I20260619 15:02:11.509210 1576 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 15:02:11.512573 1591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:11.513999 1576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 15:02:11.514148 1576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "fd4ba60ca4824e35a1a514752bfba20e"
format_stamp: "Formatted at 2026-06-19 15:02:11 on dist-test-slave-76rk"
server_key: "930bdf795ac563fd2ca9c8f6fdb7c7a9"
server_key_iv: "eed45e7fcb16f3d10ad472ba39067ded"
server_key_version: "encryptionkey@0"
I20260619 15:02:11.514325 1576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:11.527348 1576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:11.531303 1576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:11.531543 1576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:11.550889 1576 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:38281
I20260619 15:02:11.550895 1642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:38281 every 8 connection(s)
I20260619 15:02:11.552361 1576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:02:11.554690 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 1576
I20260619 15:02:11.554884 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:02:11.555418 11967 external_mini_cluster.cc:1428] Setting key b921f55370ef49d70683e2dcd79ded83
I20260619 15:02:11.556737 1643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 15:02:11 dist-test-slave-76rk krb5kdc[1563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:02:11.571343 1643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e: Bootstrap starting.
I20260619 15:02:11.574682 1643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:11.576086 1643 log.cc:824] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:11.579546 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:11.557211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56751 (local address 127.11.175.254:38281)
0619 15:02:11.558071 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:11.558083 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:11.558122 (+ 39us) server_negotiation.cc:407] Connection header received
0619 15:02:11.559802 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:11.559824 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:11.560216 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:11.560572 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:11.562330 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:11.563756 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:11.565541 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:11.566191 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:11.566349 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:11.571399 (+ 5050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:11.571407 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:11.571424 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:11.571466 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:11.574759 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:11.575724 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:11.575731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:11.575737 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:11.575932 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:11.576435 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:11.576438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:11.576440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:11.577275 (+ 835us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:11.577612 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:11.577990 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 15:02:11.578394 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":373,"threads_started":1}
I20260619 15:02:11.579596 1643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e: No bootstrap required, opened a new log
I20260619 15:02:11.583379 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:02:11.587661 1643 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4ba60ca4824e35a1a514752bfba20e" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38281 } }
I20260619 15:02:11.588140 1643 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:02:11.588271 1643 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4ba60ca4824e35a1a514752bfba20e, State: Initialized, Role: FOLLOWER
I20260619 15:02:11.588956 1643 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4ba60ca4824e35a1a514752bfba20e" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38281 } }
I20260619 15:02:11.589223 1643 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:02:11.589380 1643 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:02:11.589558 1643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:11.592468 1643 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4ba60ca4824e35a1a514752bfba20e" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38281 } }
I20260619 15:02:11.593097 1643 leader_election.cc:302] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd4ba60ca4824e35a1a514752bfba20e; no voters:
I20260619 15:02:11.593811 1643 leader_election.cc:288] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:02:11.593957 1653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:02:11.596060 1643 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:11.597577 1653 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [term 1 LEADER]: Becoming Leader. State: Replica: fd4ba60ca4824e35a1a514752bfba20e, State: Running, Role: LEADER
I20260619 15:02:11.598209 1653 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4ba60ca4824e35a1a514752bfba20e" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38281 } }
I20260619 15:02:11.601581 1655 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd4ba60ca4824e35a1a514752bfba20e. Latest consensus state: current_term: 1 leader_uuid: "fd4ba60ca4824e35a1a514752bfba20e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4ba60ca4824e35a1a514752bfba20e" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38281 } } }
I20260619 15:02:11.602001 1655 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:11.602479 1654 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd4ba60ca4824e35a1a514752bfba20e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4ba60ca4824e35a1a514752bfba20e" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 38281 } } }
I20260619 15:02:11.602818 1654 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:11.607697 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:11.587798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44869 (local address 127.11.175.254:38281)
0619 15:02:11.587984 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:11.587991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:11.588472 (+ 481us) server_negotiation.cc:407] Connection header received
0619 15:02:11.588665 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:11.588670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:11.588745 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:11.588852 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:11.590412 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:11.591586 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:11.592889 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:11.593581 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:11.593735 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:11.597390 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:11.597401 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:11.597409 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:11.597468 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:11.603149 (+ 5681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:11.604217 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:11.604223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:11.604232 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:11.604472 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:11.605079 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:11.605087 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:11.605093 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:11.605561 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:11.607167 (+ 1606us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:11.607284 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 15:02:11.607440 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 15:02:11.612003 1669 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 15:02:11.612143 1669 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 15:02:11.612299 1661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:02:11.614075 1661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:02:11.623728 1661 catalog_manager.cc:1347] Generated new cluster ID: f70e0a87c17b4bcf975cb73ce68d6cc4
I20260619 15:02:11.623868 1661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:02:11.638286 1661 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:02:11.639802 1661 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:02:11.650027 1661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd4ba60ca4824e35a1a514752bfba20e: Generated new TSK 0
I20260619 15:02:11.651189 1661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:02:11.659389 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1576
I20260619 15:02:11.661056 1638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:02:11.751801 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 1576
2026-06-19T15:02:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2029 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:02:11 dist-test-slave-76rk krb5kdc[1675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:02:11 dist-test-slave-76rk krb5kdc[1675](info): set up 2 sockets
Jun 19 15:02:11 dist-test-slave-76rk krb5kdc[1675](info): commencing operation
krb5kdc: starting...
W20260619 15:02:12.891475 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:02:13 dist-test-slave-76rk krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:02:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:02:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:02:13.169636 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:36351
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:36351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:02:13.514919 1688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:13.515322 1688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:13.515496 1688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:13.523092 1688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:13.523258 1688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:13.523375 1688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:13.523486 1688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:13.523571 1688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:13.533890 1688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:36351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:36351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:13.535341 1688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:13.536773 1688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:13.549496 1693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:13.549422 1696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:13.549460 1694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:13.550977 1688 server_base.cc:1034] running on GCE node
I20260619 15:02:13.551883 1688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:13.553423 1688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:13.554724 1688 hybrid_clock.cc:648] HybridClock initialized: now 1781881333554592 us; error 65 us; skew 500 ppm
I20260619 15:02:13.555182 1688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:13 dist-test-slave-76rk krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:13.561815 1688 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:02:13.563550 1688 webserver.cc:460] Webserver started at http://127.11.175.254:36609/ using document root <none> and password file <none>
I20260619 15:02:13.564267 1688 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:13.564371 1688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:13.564630 1688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:13.567859 1688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "19082edc55d3429b988b08de5ad1fc87"
format_stamp: "Formatted at 2026-06-19 15:02:13 on dist-test-slave-76rk"
server_key: "c6cea036907b05887c3c91eb37281e36"
server_key_iv: "6a5a8f9d7026fd6d48892f9f94592116"
server_key_version: "encryptionkey@0"
I20260619 15:02:13.568622 1688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "19082edc55d3429b988b08de5ad1fc87"
format_stamp: "Formatted at 2026-06-19 15:02:13 on dist-test-slave-76rk"
server_key: "c6cea036907b05887c3c91eb37281e36"
server_key_iv: "6a5a8f9d7026fd6d48892f9f94592116"
server_key_version: "encryptionkey@0"
I20260619 15:02:13.574236 1688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 15:02:13.578205 1703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:13.579931 1688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 15:02:13.580163 1688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "19082edc55d3429b988b08de5ad1fc87"
format_stamp: "Formatted at 2026-06-19 15:02:13 on dist-test-slave-76rk"
server_key: "c6cea036907b05887c3c91eb37281e36"
server_key_iv: "6a5a8f9d7026fd6d48892f9f94592116"
server_key_version: "encryptionkey@0"
I20260619 15:02:13.580390 1688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:13.613317 1688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:13.618579 1688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:13.618992 1688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:13.640695 1688 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:36351
I20260619 15:02:13.640797 1754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:36351 every 8 connection(s)
I20260619 15:02:13.642269 1688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:02:13.646364 1755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:13.647553 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 1688
I20260619 15:02:13.647763 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:02:13.648330 11967 external_mini_cluster.cc:1428] Setting key ece48a1cba512fa25616bbc11d02341c
I20260619 15:02:13.661972 1755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87: Bootstrap starting.
I20260619 15:02:13.663951 1758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:13.650392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46375 (local address 127.11.175.254:36351)
0619 15:02:13.651335 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:13.651348 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:13.651398 (+ 50us) server_negotiation.cc:407] Connection header received
0619 15:02:13.653158 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:13.653186 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:13.653675 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:13.654093 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:13.654812 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:13.656109 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:13.658136 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:13.659043 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:13.659185 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:13.660744 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:13.660755 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:13.660776 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:13.661681 (+ 905us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:13.661807 (+ 126us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:13.661844 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:13.661976 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:13.662422 (+ 446us) server_negotiation.cc:299] Negotiation successful
0619 15:02:13.662872 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":484,"threads_started":1}
I20260619 15:02:13.665580 1755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:13.666846 1755 log.cc:824] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:13.668058 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:02:13.670543 1755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87: No bootstrap required, opened a new log
I20260619 15:02:13.680738 1755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19082edc55d3429b988b08de5ad1fc87" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36351 } }
I20260619 15:02:13.681329 1755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:02:13.681488 1755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19082edc55d3429b988b08de5ad1fc87, State: Initialized, Role: FOLLOWER
I20260619 15:02:13.681941 1758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:13.673924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33125 (local address 127.11.175.254:36351)
0619 15:02:13.674144 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:13.674151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:13.674884 (+ 733us) server_negotiation.cc:407] Connection header received
0619 15:02:13.675074 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:13.675080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:13.675171 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:13.675302 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:13.675876 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:13.677052 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:13.678721 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:13.679494 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:13.679648 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:13.680993 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:13.681000 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:13.681004 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:13.681029 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:13.681081 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:13.681104 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:13.681215 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:13.681473 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 15:02:13.681576 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 15:02:13.682260 1755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19082edc55d3429b988b08de5ad1fc87" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36351 } }
I20260619 15:02:13.682564 1755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:02:13.682793 1755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:02:13.682994 1755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:13.685626 1755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19082edc55d3429b988b08de5ad1fc87" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36351 } }
I20260619 15:02:13.686213 1755 leader_election.cc:302] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19082edc55d3429b988b08de5ad1fc87; no voters:
I20260619 15:02:13.686934 1755 leader_election.cc:288] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:02:13.687049 1765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:02:13.687793 1765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [term 1 LEADER]: Becoming Leader. State: Replica: 19082edc55d3429b988b08de5ad1fc87, State: Running, Role: LEADER
I20260619 15:02:13.688381 1765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19082edc55d3429b988b08de5ad1fc87" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36351 } }
I20260619 15:02:13.689541 1755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:13.691542 1767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19082edc55d3429b988b08de5ad1fc87. Latest consensus state: current_term: 1 leader_uuid: "19082edc55d3429b988b08de5ad1fc87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19082edc55d3429b988b08de5ad1fc87" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36351 } } }
I20260619 15:02:13.691965 1767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:13.697239 1766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19082edc55d3429b988b08de5ad1fc87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19082edc55d3429b988b08de5ad1fc87" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 36351 } } }
I20260619 15:02:13.697686 1766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:13.699630 1772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:02:13.707001 1772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:02:13.718360 1772 catalog_manager.cc:1347] Generated new cluster ID: b02b6666a1194347958cfbec849b6876
I20260619 15:02:13.718502 1772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:02:13.738076 1772 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:02:13.739578 1772 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:02:13.750054 1772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19082edc55d3429b988b08de5ad1fc87: Generated new TSK 0
I20260619 15:02:13.751220 1772 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:02:13.764482 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1688
W20260619 15:02:13.766021 1750 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46375
W20260619 15:02:13.766897 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1688: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:02:13.767088 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 1688
2026-06-19T15:02:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2014 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:02:13 dist-test-slave-76rk krb5kdc[1786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:02:13 dist-test-slave-76rk krb5kdc[1786](info): set up 2 sockets
Jun 19 15:02:13 dist-test-slave-76rk krb5kdc[1786](info): commencing operation
krb5kdc: starting...
W20260619 15:02:14.907232 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.006s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:02:15 dist-test-slave-76rk krb5kdc[1786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:02:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:02:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:02:15.190415 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:46755
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:34599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:46755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:02:15.531631 1799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:15.531921 1799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:15.532089 1799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:15.539517 1799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:15.539646 1799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:15.539742 1799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:15.539858 1799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:15.539927 1799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:15.549966 1799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:34599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:46755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:46755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:15.551345 1799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:15.552695 1799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:15.565062 1805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:15.565399 1804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:15.565462 1807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:15.566241 1799 server_base.cc:1034] running on GCE node
I20260619 15:02:15.567221 1799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:15.568754 1799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:15.569976 1799 hybrid_clock.cc:648] HybridClock initialized: now 1781881335569879 us; error 92 us; skew 500 ppm
I20260619 15:02:15.570401 1799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:15 dist-test-slave-76rk krb5kdc[1786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:15.573904 1799 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:02:15.575747 1799 webserver.cc:460] Webserver started at http://127.11.175.254:35837/ using document root <none> and password file <none>
I20260619 15:02:15.576531 1799 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:15.576704 1799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:15.577072 1799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:15.580579 1799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "663460b85c0e42679f8a9a535dec7f36"
format_stamp: "Formatted at 2026-06-19 15:02:15 on dist-test-slave-76rk"
server_key: "b730443f2379b2b4dee042c5db6e3d64"
server_key_iv: "4acb422a895c0aec62d143ec3909668e"
server_key_version: "encryptionkey@0"
I20260619 15:02:15.581542 1799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "663460b85c0e42679f8a9a535dec7f36"
format_stamp: "Formatted at 2026-06-19 15:02:15 on dist-test-slave-76rk"
server_key: "b730443f2379b2b4dee042c5db6e3d64"
server_key_iv: "4acb422a895c0aec62d143ec3909668e"
server_key_version: "encryptionkey@0"
I20260619 15:02:15.587685 1799 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260619 15:02:15.591792 1814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:15.593562 1799 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 15:02:15.593783 1799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "663460b85c0e42679f8a9a535dec7f36"
format_stamp: "Formatted at 2026-06-19 15:02:15 on dist-test-slave-76rk"
server_key: "b730443f2379b2b4dee042c5db6e3d64"
server_key_iv: "4acb422a895c0aec62d143ec3909668e"
server_key_version: "encryptionkey@0"
I20260619 15:02:15.593990 1799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:15.600498 1799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:15.605293 1799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:15.605644 1799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:15.626456 1799 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:46755
I20260619 15:02:15.626493 1865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:46755 every 8 connection(s)
I20260619 15:02:15.628046 1799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:02:15.628373 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 1799
I20260619 15:02:15.628547 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:02:15.629245 11967 external_mini_cluster.cc:1428] Setting key 9d1a6e150953989ef4ca68eff144174e
I20260619 15:02:15.632788 1866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:15.638243 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:15.631277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39747 (local address 127.11.175.254:46755)
0619 15:02:15.632159 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:15.632173 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:15.632218 (+ 45us) server_negotiation.cc:407] Connection header received
0619 15:02:15.634156 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:15.634181 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:15.634555 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:15.635089 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:15.636345 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:15.636352 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:15.636370 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:15.636389 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:15.636480 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:15.636505 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:15.636819 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:15.636995 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 15:02:15.637197 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":374,"threads_started":1}
I20260619 15:02:15.642848 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:02:15.644896 1866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36: Bootstrap starting.
I20260619 15:02:15.648921 1866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:15.650477 1866 log.cc:824] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:15.650542 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:15.647518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57519 (local address 127.11.175.254:46755)
0619 15:02:15.647769 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:15.647778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:15.648184 (+ 406us) server_negotiation.cc:407] Connection header received
0619 15:02:15.648391 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:15.648394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:15.648450 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:15.648528 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:15.649682 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:15.649687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:15.649691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 15:02:15.649709 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:15.649755 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 15:02:15.649784 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:15.649895 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:15.650090 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 15:02:15.650144 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 15:02:15.653633 1866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36: No bootstrap required, opened a new log
I20260619 15:02:15.661767 1866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663460b85c0e42679f8a9a535dec7f36" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 46755 } }
I20260619 15:02:15.662222 1866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 15:02:15.662346 1866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663460b85c0e42679f8a9a535dec7f36, State: Initialized, Role: FOLLOWER
I20260619 15:02:15.663051 1866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663460b85c0e42679f8a9a535dec7f36" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 46755 } }
I20260619 15:02:15.663327 1866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 15:02:15.663479 1866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 15:02:15.663661 1866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:15.665967 1866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663460b85c0e42679f8a9a535dec7f36" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 46755 } }
I20260619 15:02:15.666551 1866 leader_election.cc:302] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 663460b85c0e42679f8a9a535dec7f36; no voters:
I20260619 15:02:15.667361 1866 leader_election.cc:288] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 15:02:15.667544 1876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:02:15.671443 1866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:15.672169 1876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [term 1 LEADER]: Becoming Leader. State: Replica: 663460b85c0e42679f8a9a535dec7f36, State: Running, Role: LEADER
I20260619 15:02:15.672793 1876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663460b85c0e42679f8a9a535dec7f36" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 46755 } }
I20260619 15:02:15.675962 1878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 663460b85c0e42679f8a9a535dec7f36. Latest consensus state: current_term: 1 leader_uuid: "663460b85c0e42679f8a9a535dec7f36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663460b85c0e42679f8a9a535dec7f36" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 46755 } } }
I20260619 15:02:15.675872 1877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "663460b85c0e42679f8a9a535dec7f36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663460b85c0e42679f8a9a535dec7f36" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 46755 } } }
I20260619 15:02:15.677246 1877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:15.677104 1878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:15.679682 1886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:02:15.684681 1886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:02:15.694873 1886 catalog_manager.cc:1347] Generated new cluster ID: 4a40f05add014f8e8184946539ea5353
I20260619 15:02:15.694999 1886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:02:15.746171 1886 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:02:15.746709 1886 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:02:15.757033 1886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 663460b85c0e42679f8a9a535dec7f36: Generated new TSK 0
I20260619 15:02:15.757931 1886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:02:15.766037 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1799
W20260619 15:02:15.767237 1861 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39747
W20260619 15:02:15.767943 11967 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1799: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 15:02:15.768107 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 1799
2026-06-19T15:02:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2000 ms)
[----------] 18 tests from AuthTokenIssuingTest (37135 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:02:15 dist-test-slave-76rk krb5kdc[1897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:02:15 dist-test-slave-76rk krb5kdc[1897](info): set up 2 sockets
Jun 19 15:02:15 dist-test-slave-76rk krb5kdc[1897](info): commencing operation
krb5kdc: starting...
W20260619 15:02:16.923384 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:02:17 dist-test-slave-76rk krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:02:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:02:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:02:17.207756 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:42085
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:42085,127.11.175.253:35165,127.11.175.252:42011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:02:17.556594 1910 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 15:02:17.556944 1910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:17.557089 1910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:17.557229 1910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:17.564764 1910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:17.564873 1910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:17.564967 1910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:17.565047 1910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:17.565145 1910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:17.575601 1910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:42085,127.11.175.253:35165,127.11.175.252:42011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:42085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:17.577008 1910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:17.578413 1910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:17.591272 1915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:17.591420 1916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:17.591485 1918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:17.592504 1910 server_base.cc:1034] running on GCE node
I20260619 15:02:17.593313 1910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:17.594952 1910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:17.596196 1910 hybrid_clock.cc:648] HybridClock initialized: now 1781881337596087 us; error 79 us; skew 500 ppm
I20260619 15:02:17.596575 1910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:17 dist-test-slave-76rk krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:17.600157 1910 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:02:17.601742 1910 webserver.cc:460] Webserver started at http://127.11.175.254:35463/ using document root <none> and password file <none>
I20260619 15:02:17.602494 1910 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:17.602628 1910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:17.603012 1910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:17.606142 1910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "513513cc7a9b44a98eb29836f2fb2b80"
format_stamp: "Formatted at 2026-06-19 15:02:17 on dist-test-slave-76rk"
server_key: "c24884607fdf1be6cf78f8bacf0f1c19"
server_key_iv: "56773a69a3e59d20da64ae8a4b62d58d"
server_key_version: "encryptionkey@0"
I20260619 15:02:17.607024 1910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "513513cc7a9b44a98eb29836f2fb2b80"
format_stamp: "Formatted at 2026-06-19 15:02:17 on dist-test-slave-76rk"
server_key: "c24884607fdf1be6cf78f8bacf0f1c19"
server_key_iv: "56773a69a3e59d20da64ae8a4b62d58d"
server_key_version: "encryptionkey@0"
I20260619 15:02:17.612320 1910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 15:02:17.616057 1925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:17.617456 1910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 15:02:17.617643 1910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "513513cc7a9b44a98eb29836f2fb2b80"
format_stamp: "Formatted at 2026-06-19 15:02:17 on dist-test-slave-76rk"
server_key: "c24884607fdf1be6cf78f8bacf0f1c19"
server_key_iv: "56773a69a3e59d20da64ae8a4b62d58d"
server_key_version: "encryptionkey@0"
I20260619 15:02:17.617877 1910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:17.645512 1910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:17.650390 1910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:17.650732 1910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:17.670306 1910 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:42085
I20260619 15:02:17.670326 1976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:42085 every 8 connection(s)
I20260619 15:02:17.671739 1910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:02:17.675619 1977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:17.677536 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 1910
I20260619 15:02:17.677779 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:02:17.678787 11967 external_mini_cluster.cc:1428] Setting key e862ae4a55f531cce552d290e5253633
I20260619 15:02:17.683347 1977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 15:02:17 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.175.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.253@KRBTEST.COM" created.
I20260619 15:02:17.711462 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:17.686787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42085 (local address 127.0.0.1:45666)
0619 15:02:17.687815 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:17.687872 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:17.690937 (+ 3065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:17.692038 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:17.692048 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:17.692909 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:17.693453 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:17.693485 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:17.695421 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:17.695429 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:17.697439 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:17.697458 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:17.698562 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:17.698572 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:17.698820 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:17.699607 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:17.699642 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:17.702943 (+ 3301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:17.706595 (+ 3652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:17.706606 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:17.706617 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:17.707168 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:17.707570 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:17.707573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:17.707576 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:17.707743 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:17.708664 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:17.708675 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:17.708950 (+ 275us) client_negotiation.cc:769] Sending connection context
0619 15:02:17.709213 (+ 263us) client_negotiation.cc:240] Negotiation successful
0619 15:02:17.709932 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":189,"thread_start_us":403,"threads_started":1}
I20260619 15:02:17.711491 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:17.686787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45666 (local address 127.11.175.254:42085)
0619 15:02:17.688877 (+ 2090us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:17.688885 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:17.688931 (+ 46us) server_negotiation.cc:407] Connection header received
0619 15:02:17.691436 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:17.691443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:17.691757 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:17.691932 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:17.693740 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:17.695244 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:17.697608 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:17.698284 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:17.698425 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:17.703219 (+ 4794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:17.703228 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:17.703242 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:17.703310 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:17.706452 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:17.707302 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:17.707309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:17.707312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:17.707464 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:17.707855 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:17.707858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:17.707860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:17.708417 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:17.708582 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:17.709364 (+ 782us) server_negotiation.cc:299] Negotiation successful
0619 15:02:17.709802 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":1879,"spinlock_wait_cycles":26880,"thread_start_us":1619,"threads_started":1}
I20260619 15:02:17.715085 1977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } has no permanent_uuid. Determining permanent_uuid...
I20260619 15:02:17.716370 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:17.715922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43296)
0619 15:02:17.716100 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:17.716219 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260619 15:02:17.716697 1929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
I20260619 15:02:17.718286 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:17.717926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43300)
0619 15:02:17.718078 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:17.718147 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 15:02:17.718505 1977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:35165: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab.
Entry for principal kudu/127.11.175.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab.
I20260619 15:02:17.741792 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.253:35165
--webserver_interface=127.11.175.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab
--principal=kudu/127.11.175.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:42085,127.11.175.253:35165,127.11.175.252:42011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 15:02:17.767843 1977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } attempt: 1
I20260619 15:02:17.769443 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:17.768860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43314)
0619 15:02:17.769096 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:17.769277 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 15:02:17.770530 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:17.770130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43322)
0619 15:02:17.770290 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:17.770380 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260619 15:02:17.770792 1977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:35165: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
I20260619 15:02:17.839111 1977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } attempt: 2
I20260619 15:02:17.840588 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:17.839959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43328)
0619 15:02:17.840256 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:17.840399 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 15:02:17.841820 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:17.841381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43336)
0619 15:02:17.841557 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:17.841649 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 15:02:17.842065 1977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:35165: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
I20260619 15:02:17.933357 1977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } attempt: 3
I20260619 15:02:17.934860 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:17.934258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43348)
0619 15:02:17.934524 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:17.934688 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260619 15:02:17.936048 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:17.935590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43358)
0619 15:02:17.935775 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:17.935871 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260619 15:02:17.936311 1977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:35165: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
I20260619 15:02:18.079582 1977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } attempt: 4
I20260619 15:02:18.080931 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.080323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43368)
0619 15:02:18.080675 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.080790 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260619 15:02:18.082026 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.081652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43382)
0619 15:02:18.081827 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.081908 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260619 15:02:18.082213 1977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:35165: Network error: Client connection negotiation failed: client connection to 127.11.175.253:35165: connect: Connection refused (error 111)
W20260619 15:02:18.098866 1984 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 15:02:18.099291 1984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:18.099431 1984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:18.099586 1984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:18.107926 1984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:18.108062 1984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:18.108198 1984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:18.108287 1984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:18.108399 1984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:18.118775 1984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:42085,127.11.175.253:35165,127.11.175.252:42011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab
--principal=kudu/127.11.175.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.253:35165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.175.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:18.120119 1984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:18.121552 1984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:18.133919 1990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:18.134007 1989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:18.134007 1992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:18.135022 1984 server_base.cc:1034] running on GCE node
I20260619 15:02:18.135866 1984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:18.137315 1984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:18.138562 1984 hybrid_clock.cc:648] HybridClock initialized: now 1781881338138484 us; error 56 us; skew 500 ppm
I20260619 15:02:18.138931 1984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881338, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:18.142558 1984 init.cc:377] Logged in from keytab as kudu/127.11.175.253@KRBTEST.COM (short username kudu)
I20260619 15:02:18.144157 1984 webserver.cc:460] Webserver started at http://127.11.175.253:45463/ using document root <none> and password file <none>
I20260619 15:02:18.144878 1984 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:18.145010 1984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:18.145339 1984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:18.148334 1984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/data/instance:
uuid: "f2d9f3672be8433098e3498c5e7867b6"
format_stamp: "Formatted at 2026-06-19 15:02:18 on dist-test-slave-76rk"
server_key: "cf4cd1b83f3dd9c12d0becd24b13b2a0"
server_key_iv: "fd2071c8cd42bbaadfd4fe1c76215515"
server_key_version: "encryptionkey@0"
I20260619 15:02:18.149156 1984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/wal/instance:
uuid: "f2d9f3672be8433098e3498c5e7867b6"
format_stamp: "Formatted at 2026-06-19 15:02:18 on dist-test-slave-76rk"
server_key: "cf4cd1b83f3dd9c12d0becd24b13b2a0"
server_key_iv: "fd2071c8cd42bbaadfd4fe1c76215515"
server_key_version: "encryptionkey@0"
I20260619 15:02:18.154410 1984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 15:02:18.157998 1999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:18.159476 1984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 15:02:18.159680 1984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/wal
uuid: "f2d9f3672be8433098e3498c5e7867b6"
format_stamp: "Formatted at 2026-06-19 15:02:18 on dist-test-slave-76rk"
server_key: "cf4cd1b83f3dd9c12d0becd24b13b2a0"
server_key_iv: "fd2071c8cd42bbaadfd4fe1c76215515"
server_key_version: "encryptionkey@0"
I20260619 15:02:18.159924 1984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:18.195158 1984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:18.199852 1984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:18.200088 1984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:18.219542 1984 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.253:35165
I20260619 15:02:18.219548 2050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.253:35165 every 8 connection(s)
I20260619 15:02:18.221012 1984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
I20260619 15:02:18.224959 2051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:18.230294 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 1984
I20260619 15:02:18.230499 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-1/wal/instance
I20260619 15:02:18.230978 11967 external_mini_cluster.cc:1428] Setting key e566fb921517f3eb0721c6f86139988a
I20260619 15:02:18.232051 2051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881338, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:02:18.255831 2053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.234922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.11.175.254:42085)
0619 15:02:18.235456 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.235461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.236467 (+ 1006us) server_negotiation.cc:407] Connection header received
0619 15:02:18.238408 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.238415 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.238527 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.238701 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.240697 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.241846 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.243526 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.244248 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.244392 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.247995 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.248004 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.248012 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.248076 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.251858 (+ 3782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.252752 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.252760 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.252766 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.252925 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.253446 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.253454 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.253460 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.253866 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.254035 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.255037 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.255313 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":315,"threads_started":1}
I20260619 15:02:18.256445 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.235474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42085 (local address 127.0.0.1:45680)
0619 15:02:18.236256 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.236332 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:18.238111 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:18.238955 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:18.238964 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:18.239521 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:18.240428 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.240456 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.242032 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.242036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.243374 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.243387 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.244481 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.244485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.244689 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.245400 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:18.245424 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:18.247718 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:18.252045 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.252052 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.252067 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.252602 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.253079 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.253085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.253088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.253301 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.254031 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:18.254038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:18.254543 (+ 505us) client_negotiation.cc:769] Sending connection context
0619 15:02:18.254873 (+ 330us) client_negotiation.cc:240] Negotiation successful
0619 15:02:18.255290 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":326,"threads_started":1}
I20260619 15:02:18.258462 2051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.175.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.252@KRBTEST.COM" created.
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881338, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 15:02:18.276124 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.259627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.11.175.253:35165)
0619 15:02:18.260163 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.260168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.260193 (+ 25us) server_negotiation.cc:407] Connection header received
0619 15:02:18.260239 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.260245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.260488 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.260631 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.261620 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.263522 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.264811 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.265718 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.265854 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.269387 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.269407 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.269437 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.269480 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.272913 (+ 3433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.273852 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.273858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.273860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.273979 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.274405 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.274408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.274410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.274860 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.275024 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.275476 (+ 452us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.275825 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":399,"thread_start_us":283,"threads_started":1}
I20260619 15:02:18.276216 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.259482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43388)
0619 15:02:18.259687 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.259708 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:18.259842 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:18.260684 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:18.260687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:18.261187 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:18.261408 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.261423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.263661 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.263665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.264647 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.264658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.265880 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.265885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.266087 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.266806 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:18.266823 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:18.269142 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:18.273108 (+ 3966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.273116 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.273124 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.273689 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.274093 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.274101 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.274107 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.274298 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.274994 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:18.275001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:18.275181 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 15:02:18.275354 (+ 173us) client_negotiation.cc:240] Negotiation successful
0619 15:02:18.275702 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":39}
I20260619 15:02:18.279088 2051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } has no permanent_uuid. Determining permanent_uuid...
I20260619 15:02:18.280455 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.279879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47552)
0619 15:02:18.280184 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.280292 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260619 15:02:18.280804 2000 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
I20260619 15:02:18.282450 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.282078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47560)
0619 15:02:18.282261 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.282338 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 15:02:18.282768 2051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:42011: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab.
Entry for principal kudu/127.11.175.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab.
I20260619 15:02:18.299072 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.252:42011
--webserver_interface=127.11.175.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab
--principal=kudu/127.11.175.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:42085,127.11.175.253:35165,127.11.175.252:42011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 15:02:18.332122 2051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } attempt: 1
I20260619 15:02:18.333320 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.332833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47564)
0619 15:02:18.333037 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.333147 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260619 15:02:18.334303 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.333970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47568)
0619 15:02:18.334109 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.334189 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 15:02:18.334549 2051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:42011: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
I20260619 15:02:18.381497 1977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } attempt: 5
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 15:02:18.397027 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.382385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43392 (local address 127.11.175.253:35165)
0619 15:02:18.382560 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.382564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.382583 (+ 19us) server_negotiation.cc:407] Connection header received
0619 15:02:18.382799 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.382803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.382888 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.382975 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.383970 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.384713 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.386444 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.387069 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.387186 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.391291 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.391297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.391300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.391336 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.394115 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.394940 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.394946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.394952 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.395083 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.395504 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.395508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.395510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.395807 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.395978 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.396421 (+ 443us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.396656 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 15:02:18.397027 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.382248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43392)
0619 15:02:18.382482 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.382505 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:18.382665 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:18.383047 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:18.383052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:18.383556 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:18.383789 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.383803 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.384860 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.384866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.386257 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.386274 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.387240 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.387246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.387471 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.388169 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:18.388192 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:18.391046 (+ 2854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:18.394303 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.394309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.394313 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.394811 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.395212 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.395218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.395221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.395374 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.395954 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:18.395958 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:18.396143 (+ 185us) client_negotiation.cc:769] Sending connection context
0619 15:02:18.396297 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 15:02:18.396656 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260619 15:02:18.398275 1977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } has no permanent_uuid. Determining permanent_uuid...
I20260619 15:02:18.399555 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.399043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47580)
0619 15:02:18.399269 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.399367 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260619 15:02:18.400605 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.400187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47586)
0619 15:02:18.400378 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.400476 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260619 15:02:18.400877 1977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:42011: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
I20260619 15:02:18.402726 2051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } attempt: 2
I20260619 15:02:18.403733 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.403246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47588)
0619 15:02:18.403490 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.403582 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260619 15:02:18.405098 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.404562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47592)
0619 15:02:18.404709 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.404793 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 15:02:18.405382 2051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:42011: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
I20260619 15:02:18.452158 1977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } attempt: 1
I20260619 15:02:18.453397 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.452840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47600)
0619 15:02:18.453069 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.453195 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260619 15:02:18.454438 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.454063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47612)
0619 15:02:18.454233 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.454315 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260619 15:02:18.454684 1977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:42011: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
I20260619 15:02:18.496641 2051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } attempt: 3
I20260619 15:02:18.497938 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.497356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47622)
0619 15:02:18.497613 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.497740 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 15:02:18.499364 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.498759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47632)
0619 15:02:18.499034 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.499170 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260619 15:02:18.499642 2051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:42011: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
I20260619 15:02:18.522900 1977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } attempt: 2
I20260619 15:02:18.524103 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.523550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47644)
0619 15:02:18.523788 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.523899 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 15:02:18.525246 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.524755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47660)
0619 15:02:18.524954 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.525061 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260619 15:02:18.525492 1977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:42011: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
I20260619 15:02:18.631771 1977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } attempt: 3
I20260619 15:02:18.633085 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.632502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47672)
0619 15:02:18.632751 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.632868 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260619 15:02:18.634413 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.633891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47674)
0619 15:02:18.634111 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.634264 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260619 15:02:18.634783 1977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:42011: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
I20260619 15:02:18.642890 2051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } attempt: 4
I20260619 15:02:18.644137 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.643563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47690)
0619 15:02:18.643838 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.643959 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260619 15:02:18.645231 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.644865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47696)
0619 15:02:18.645012 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.645101 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 15:02:18.645498 2051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:42011: Network error: Client connection negotiation failed: client connection to 127.11.175.252:42011: connect: Connection refused (error 111)
W20260619 15:02:18.646150 2059 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 15:02:18.646512 2059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:18.646615 2059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:18.646819 2059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:18.654398 2059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:18.654500 2059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:18.654605 2059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:18.654723 2059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:18.654815 2059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:18.665096 2059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:42085,127.11.175.253:35165,127.11.175.252:42011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab
--principal=kudu/127.11.175.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.252:42011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.175.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:18.666482 2059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:18.667860 2059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:18.683248 2065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:18.683476 2067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:18.683910 2064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:18.684403 2059 server_base.cc:1034] running on GCE node
I20260619 15:02:18.685288 2059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:18.686940 2059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:18.688231 2059 hybrid_clock.cc:648] HybridClock initialized: now 1781881338688126 us; error 33 us; skew 500 ppm
I20260619 15:02:18.688629 2059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881338, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:18.697510 2059 init.cc:377] Logged in from keytab as kudu/127.11.175.252@KRBTEST.COM (short username kudu)
I20260619 15:02:18.699398 2059 webserver.cc:460] Webserver started at http://127.11.175.252:33625/ using document root <none> and password file <none>
I20260619 15:02:18.700181 2059 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:18.700294 2059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:18.700587 2059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:18.704057 2059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/data/instance:
uuid: "f5d50a993a164f1d832d28ca9f9c1601"
format_stamp: "Formatted at 2026-06-19 15:02:18 on dist-test-slave-76rk"
server_key: "71de8b671018322ad4a2fd61681e2f3b"
server_key_iv: "f7938cfdf469d473bdd2557d72fc8a9d"
server_key_version: "encryptionkey@0"
I20260619 15:02:18.704918 2059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/wal/instance:
uuid: "f5d50a993a164f1d832d28ca9f9c1601"
format_stamp: "Formatted at 2026-06-19 15:02:18 on dist-test-slave-76rk"
server_key: "71de8b671018322ad4a2fd61681e2f3b"
server_key_iv: "f7938cfdf469d473bdd2557d72fc8a9d"
server_key_version: "encryptionkey@0"
I20260619 15:02:18.711236 2059 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 15:02:18.715518 2074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:18.717236 2059 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 15:02:18.717471 2059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/wal
uuid: "f5d50a993a164f1d832d28ca9f9c1601"
format_stamp: "Formatted at 2026-06-19 15:02:18 on dist-test-slave-76rk"
server_key: "71de8b671018322ad4a2fd61681e2f3b"
server_key_iv: "f7938cfdf469d473bdd2557d72fc8a9d"
server_key_version: "encryptionkey@0"
I20260619 15:02:18.717737 2059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:18.732003 2059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:18.737030 2059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:18.737360 2059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:18.756459 2059 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.252:42011
I20260619 15:02:18.756532 2125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.252:42011 every 8 connection(s)
I20260619 15:02:18.757906 2059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
I20260619 15:02:18.762197 2126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:18.766444 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 2059
I20260619 15:02:18.766667 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781880972997616-11967-0/minicluster-data/master-2/wal/instance
I20260619 15:02:18.767228 11967 external_mini_cluster.cc:1428] Setting key 5bf4a14d3a321800fe88d74b42340511
I20260619 15:02:18.770612 2126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881338, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:02:18.792102 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.775241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.11.175.254:42085)
0619 15:02:18.775978 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.775988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.776413 (+ 425us) server_negotiation.cc:407] Connection header received
0619 15:02:18.776663 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.776667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.776777 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.777045 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.778359 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.779136 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.780583 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.781015 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.781169 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.785595 (+ 4426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.785603 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.785608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.785670 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.789087 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.789902 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.789908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.789912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.790060 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.790472 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.790479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.790482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.790844 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.791007 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.791524 (+ 517us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.791764 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":304,"threads_started":1}
I20260619 15:02:18.793107 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.769293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47700 (local address 127.11.175.252:42011)
0619 15:02:18.770198 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.770215 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.770266 (+ 51us) server_negotiation.cc:407] Connection header received
0619 15:02:18.772223 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.772262 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.772885 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.773395 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.774936 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.777013 (+ 2077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.778450 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.779218 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.779488 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.784137 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.784146 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.784160 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.784200 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.788359 (+ 4159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.789225 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.789235 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.789248 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.789446 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.789901 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.789908 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.789911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.790522 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.790869 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.791352 (+ 483us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.791962 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":660,"thread_start_us":417,"threads_started":1}
I20260619 15:02:18.793147 2131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.775216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:42085 (local address 127.0.0.1:45696)
0619 15:02:18.776255 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.776294 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:18.776504 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:18.777120 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:18.777128 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:18.777914 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:18.778177 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.778187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.779315 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.779319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.780412 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.780426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.781289 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.781297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.781459 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.782219 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:18.782263 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:18.785319 (+ 3056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:18.789286 (+ 3967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.789297 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.789306 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.789747 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.790190 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.790206 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.790213 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.790353 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.790981 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:18.790989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:18.791191 (+ 202us) client_negotiation.cc:769] Sending connection context
0619 15:02:18.791416 (+ 225us) client_negotiation.cc:240] Negotiation successful
0619 15:02:18.792134 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":303,"thread_start_us":588,"threads_started":1}
I20260619 15:02:18.795703 2126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881338, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:02:18.809832 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.796604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43402 (local address 127.11.175.253:35165)
0619 15:02:18.796809 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.796816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.797005 (+ 189us) server_negotiation.cc:407] Connection header received
0619 15:02:18.797185 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.797198 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.797262 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.797367 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.798132 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.798897 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.800076 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.800485 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.800620 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.803885 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.803890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.803893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.803925 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.806980 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.807689 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.807693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.807696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.807842 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.808235 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.808240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.808243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.808667 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.808810 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.809213 (+ 403us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.809435 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 15:02:18.809800 2131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.796604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:35165 (local address 127.0.0.1:43402)
0619 15:02:18.796885 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.796907 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:18.797049 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:18.797406 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:18.797411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:18.797820 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:18.797971 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.797980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.799036 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.799043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.799924 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.799938 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.800636 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.800641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.800863 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.801476 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:18.801493 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:18.803688 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:18.807130 (+ 3442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.807136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.807140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.807537 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.807958 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.807963 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.807967 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.808121 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.808783 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:18.808788 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:18.808951 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 15:02:18.809111 (+ 160us) client_negotiation.cc:240] Negotiation successful
0619 15:02:18.809420 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260619 15:02:18.810958 2126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } has no permanent_uuid. Determining permanent_uuid...
I20260619 15:02:18.812147 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.796663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45698 (local address 127.11.175.254:42085)
0619 15:02:18.796865 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.796872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.797118 (+ 246us) server_negotiation.cc:407] Connection header received
0619 15:02:18.797318 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.797322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.797395 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.797498 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.798753 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.799448 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.800716 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.801123 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.801259 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.805827 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.805835 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.805851 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.805888 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.809017 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.809861 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.809868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.809871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.810068 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.810531 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.810537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.810542 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.810905 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.811212 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.811535 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.811808 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 15:02:18.812057 1977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } attempt: 4
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881338, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 15:02:18.827986 2131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.811816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47712)
0619 15:02:18.811984 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.812005 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:18.812131 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:18.812583 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:18.812590 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:18.812966 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:18.813099 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.813108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.814409 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.814417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.815913 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.815924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.816644 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.816650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.816864 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.817533 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:18.817559 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:18.820408 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:18.824714 (+ 4306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.824722 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.824726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.825239 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.825694 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.825701 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.825706 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.825912 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.826739 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:18.826744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:18.826940 (+ 196us) client_negotiation.cc:769] Sending connection context
0619 15:02:18.827131 (+ 191us) client_negotiation.cc:240] Negotiation successful
0619 15:02:18.827490 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":3}
I20260619 15:02:18.828083 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.811890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47712 (local address 127.11.175.252:42011)
0619 15:02:18.812061 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.812067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.812085 (+ 18us) server_negotiation.cc:407] Connection header received
0619 15:02:18.812257 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.812262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.812329 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.812389 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.813271 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.814275 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.816044 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.816492 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.816622 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.820623 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.820629 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.820633 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.820672 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.824572 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.825376 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.825382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.825385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.825532 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.826056 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.826061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.826065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.826540 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.826752 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.827252 (+ 500us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.827708 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":68}
I20260619 15:02:18.831521 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.812789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47718)
0619 15:02:18.812946 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.812964 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:18.813053 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:18.814590 (+ 1537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:18.814595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:18.815104 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:18.815322 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.815334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.817502 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.817508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.819126 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.819144 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.820266 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.820272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.820512 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.821186 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:18.821209 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:18.823981 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:18.828633 (+ 4652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.828639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.828642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.829080 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.829603 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.829610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.829615 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.829817 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.830398 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:18.830407 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:18.830607 (+ 200us) client_negotiation.cc:769] Sending connection context
0619 15:02:18.830818 (+ 211us) client_negotiation.cc:240] Negotiation successful
0619 15:02:18.831131 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 15:02:18.831755 2132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.813372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47718 (local address 127.11.175.252:42011)
0619 15:02:18.814281 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.814289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.814313 (+ 24us) server_negotiation.cc:407] Connection header received
0619 15:02:18.814366 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.814370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.814433 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.814543 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.815507 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.817016 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.819287 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.820102 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.820240 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.824274 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.824287 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.824292 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.824343 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.828445 (+ 4102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.829258 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.829263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.829265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.829420 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.829955 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.829960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.829963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.830266 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.830379 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.830968 (+ 589us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.831262 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"spinlock_wait_cycles":27392,"thread_start_us":430,"threads_started":1}
I20260619 15:02:18.835176 2126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601: Bootstrap starting.
I20260619 15:02:18.835541 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.814935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.11.175.253:35165)
0619 15:02:18.815113 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.815117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.815690 (+ 573us) server_negotiation.cc:407] Connection header received
0619 15:02:18.815815 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.815823 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.815904 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.816012 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.817742 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.818439 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.820048 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.820760 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.820905 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.827752 (+ 6847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.827770 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.827776 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.827821 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.832489 (+ 4668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.833372 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.833378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.833381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.833532 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.833957 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.833962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.833965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.834274 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.834560 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.834912 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.835181 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 15:02:18.837494 1977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80: Bootstrap starting.
I20260619 15:02:18.840008 2126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:18.840981 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:02:18.841426 2126 log.cc:824] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:18.841563 1977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:18.842814 1977 log.cc:824] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:18.845118 2126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601: No bootstrap required, opened a new log
I20260619 15:02:18.846354 1977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80: No bootstrap required, opened a new log
I20260619 15:02:18.844664 1952 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "513513cc7a9b44a98eb29836f2fb2b80"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45698
I20260619 15:02:18.854242 1977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } }
I20260619 15:02:18.854737 1977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513513cc7a9b44a98eb29836f2fb2b80, State: Initialized, Role: FOLLOWER
I20260619 15:02:18.855300 1977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } }
I20260619 15:02:18.856179 2126 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } }
I20260619 15:02:18.856649 2126 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5d50a993a164f1d832d28ca9f9c1601, State: Initialized, Role: FOLLOWER
I20260619 15:02:18.856441 2135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } } }
I20260619 15:02:18.856933 2135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:18.857177 2126 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } }
I20260619 15:02:18.857836 1977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:18.857885 1952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 15:02:18.858106 1952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:18.858160 2136 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } } }
I20260619 15:02:18.858572 2136 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:18.859445 2126 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:18.861873 1952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } }
I20260619 15:02:18.864820 1952 leader_election.cc:288] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [CANDIDATE]: Term 1 election: Requested vote from peers f2d9f3672be8433098e3498c5e7867b6 (127.11.175.253:35165), f5d50a993a164f1d832d28ca9f9c1601 (127.11.175.252:42011)
I20260619 15:02:18.868072 2101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "513513cc7a9b44a98eb29836f2fb2b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f5d50a993a164f1d832d28ca9f9c1601"
I20260619 15:02:18.868736 2101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:18.876029 2101 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513513cc7a9b44a98eb29836f2fb2b80 in term 1.
I20260619 15:02:18.876935 1926 leader_election.cc:302] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 513513cc7a9b44a98eb29836f2fb2b80, f5d50a993a164f1d832d28ca9f9c1601; no voters:
I20260619 15:02:18.877655 2135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:02:18.891436 2135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [term 1 LEADER]: Becoming Leader. State: Replica: 513513cc7a9b44a98eb29836f2fb2b80, State: Running, Role: LEADER
I20260619 15:02:18.892302 2135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } }
I20260619 15:02:18.897091 2165 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 15:02:18.900611 2150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 513513cc7a9b44a98eb29836f2fb2b80. Latest consensus state: current_term: 1 leader_uuid: "513513cc7a9b44a98eb29836f2fb2b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } } }
I20260619 15:02:18.901274 2150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [sys.catalog]: This master's current role is: LEADER
W20260619 15:02:18.903895 1931 tablet.cc:2369] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:02:18.907115 2166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 15:02:18.902262 2059 master_runner.cc:186] Error getting master registration for 127.11.175.254:42085: OK, instance_id { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" instance_seqno: 1781881337664839 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 15:02:18.916934 2059 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 15:02:18.922585 2166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:02:18.924414 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.884534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.11.175.254:42085)
0619 15:02:18.884851 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.884858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.890242 (+ 5384us) server_negotiation.cc:407] Connection header received
0619 15:02:18.890458 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.890466 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.890585 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.890734 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.894838 (+ 4104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.896162 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.901195 (+ 5033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.902057 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.902188 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.911218 (+ 9030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.911229 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.911237 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.911299 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.916191 (+ 4892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.917354 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.917362 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.917365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.917575 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.919518 (+ 1943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.919528 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.919531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.920031 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.920346 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.923552 (+ 3206us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.923880 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 15:02:18.928774 2132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.884634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47732 (local address 127.11.175.252:42011)
0619 15:02:18.884863 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.884869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.889232 (+ 4363us) server_negotiation.cc:407] Connection header received
0619 15:02:18.889461 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.889470 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.889603 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.889752 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.893605 (+ 3853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.894676 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.903368 (+ 8692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.904330 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.904477 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.917408 (+ 12931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.917418 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.917424 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.917476 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.922128 (+ 4652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.923581 (+ 1453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.923590 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.923597 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.923811 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.925394 (+ 1583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.925404 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.925410 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.926306 (+ 896us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.926905 (+ 599us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.927894 (+ 989us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.928264 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 15:02:18.940300 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.885516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.11.175.253:35165)
0619 15:02:18.885749 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.885758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.898467 (+ 12709us) server_negotiation.cc:407] Connection header received
0619 15:02:18.899238 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.899248 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.899361 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.899442 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.918316 (+ 18874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.919567 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.921723 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.922461 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.922583 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.931925 (+ 9342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.931932 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.931937 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.931988 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.936439 (+ 4451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.937623 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.937632 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.937638 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.937900 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.938500 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.938508 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.938513 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.938984 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.939152 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.939670 (+ 518us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.939920 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 15:02:18.944569 2101 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [term 1 FOLLOWER]: Refusing update from remote peer 513513cc7a9b44a98eb29836f2fb2b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:02:18.945156 2051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } attempt: 5
I20260619 15:02:18.945593 2150 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Jun 19 15:02:18 dist-test-slave-76rk krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881338, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
I20260619 15:02:18.938481 1910 master_runner.cc:186] Error getting master registration for 127.11.175.253:35165: OK, instance_id { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" instance_seqno: 1781881338214107 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 15:02:18.955442 1910 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 15:02:18.957757 2136 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 513513cc7a9b44a98eb29836f2fb2b80. Latest consensus state: current_term: 1 leader_uuid: "513513cc7a9b44a98eb29836f2fb2b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } } }
I20260619 15:02:18.958096 2136 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:18.963171 2135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "513513cc7a9b44a98eb29836f2fb2b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } } }
I20260619 15:02:18.963502 2135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:18.963760 2170 mvcc.cc:205] Tried to move back new op lower bound from 7298585964303536128 to 7298585964109991936. Current Snapshot: MvccSnapshot[applied={T|T < 7298585964303536128}]
I20260619 15:02:18.963994 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.945682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:42011 (local address 127.0.0.1:47746)
0619 15:02:18.946212 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:18.946232 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:18.946344 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:18.946888 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:18.946894 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:18.947385 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:18.947614 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.947627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.949042 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.949048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.950397 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:18.950412 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.951652 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.951658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:18.951849 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.952630 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:18.952660 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:18.955721 (+ 3061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:18.960301 (+ 4580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.960308 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.960336 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.960970 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.961447 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:18.961454 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:18.961458 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:18.961754 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:18.962857 (+ 1103us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:18.962864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:18.963084 (+ 220us) client_negotiation.cc:769] Sending connection context
0619 15:02:18.963270 (+ 186us) client_negotiation.cc:240] Negotiation successful
0619 15:02:18.963604 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414}
I20260619 15:02:18.963814 2136 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "513513cc7a9b44a98eb29836f2fb2b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } } }
I20260619 15:02:18.964098 2136 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:18.964900 2132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:18.945998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47746 (local address 127.11.175.252:42011)
0619 15:02:18.946181 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:18.946187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:18.946451 (+ 264us) server_negotiation.cc:407] Connection header received
0619 15:02:18.946509 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:18.946513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:18.946601 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:18.946707 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:18.947766 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.948872 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.950571 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:18.951292 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:18.951414 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:18.955963 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:18.955969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:18.955974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:18.956011 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:18.960166 (+ 4155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.961111 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.961116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.961119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.961287 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:18.961950 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:18.961955 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:18.961958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:18.962489 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:18.962756 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:18.964301 (+ 1545us) server_negotiation.cc:299] Negotiation successful
0619 15:02:18.964534 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 15:02:18.972070 2051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6: Bootstrap starting.
I20260619 15:02:18.976511 2171 mvcc.cc:205] Tried to move back new op lower bound from 7298585964303536128 to 7298585964109991936. Current Snapshot: MvccSnapshot[applied={T|T < 7298585964303536128}]
I20260619 15:02:18.977355 2051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:18.978763 2051 log.cc:824] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:18.980604 2166 catalog_manager.cc:1347] Generated new cluster ID: f32eb84812f54d57a0d608ffa9a365f4
I20260619 15:02:18.980716 2166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:02:18.983645 2051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6: No bootstrap required, opened a new log
I20260619 15:02:18.995113 2051 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } }
I20260619 15:02:18.995580 2051 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d9f3672be8433098e3498c5e7867b6, State: Initialized, Role: FOLLOWER
I20260619 15:02:18.996177 2051 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } }
I20260619 15:02:18.997797 2176 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } } }
I20260619 15:02:18.998521 2176 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:18.998901 2051 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:18.998966 2026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "513513cc7a9b44a98eb29836f2fb2b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f2d9f3672be8433098e3498c5e7867b6"
I20260619 15:02:19.012915 2025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:19.017030 2025 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [term 1 FOLLOWER]: Refusing update from remote peer 513513cc7a9b44a98eb29836f2fb2b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:02:19.019295 2166 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:02:19.019687 2150 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:02:19.020448 2026 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513513cc7a9b44a98eb29836f2fb2b80 in term 1.
I20260619 15:02:19.021406 2166 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:02:19.036818 2166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80: Generated new TSK 0
I20260619 15:02:19.038111 2166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:02:19.045574 2176 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 513513cc7a9b44a98eb29836f2fb2b80. Latest consensus state: current_term: 1 leader_uuid: "513513cc7a9b44a98eb29836f2fb2b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } } }
I20260619 15:02:19.045930 2176 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:19.046168 2150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "513513cc7a9b44a98eb29836f2fb2b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } } }
I20260619 15:02:19.046105 2189 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "513513cc7a9b44a98eb29836f2fb2b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513513cc7a9b44a98eb29836f2fb2b80" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 42085 } } peers { permanent_uuid: "f2d9f3672be8433098e3498c5e7867b6" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 35165 } } peers { permanent_uuid: "f5d50a993a164f1d832d28ca9f9c1601" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 42011 } } }
I20260619 15:02:19.046432 2150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513513cc7a9b44a98eb29836f2fb2b80 [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:19.046460 2189 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:19.064333 2192 catalog_manager.cc:1259] Loaded cluster ID: f32eb84812f54d57a0d608ffa9a365f4
I20260619 15:02:19.064515 2192 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6: loading cluster ID for follower catalog manager: success
I20260619 15:02:19.068849 2192 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6: acquiring CA information for follower catalog manager: success
I20260619 15:02:19.071949 2192 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2d9f3672be8433098e3498c5e7867b6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 15:02:19.132462 2130 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.114059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45712 (local address 127.11.175.254:42085)
0619 15:02:19.114557 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.114572 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.130429 (+ 15857us) server_negotiation.cc:407] Connection header received
0619 15:02:19.130796 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.130804 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.130892 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.130976 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.132204 (+ 1228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45712: BlockingRecv error: recv got EOF from 127.0.0.1:45712 (error 108)
Metrics: {"server-negotiator.queue_time_us":330}
W20260619 15:02:19.133940 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.114311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47756 (local address 127.11.175.252:42011)
0619 15:02:19.114782 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.114800 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.131015 (+ 16215us) server_negotiation.cc:407] Connection header received
0619 15:02:19.131375 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.131384 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.131502 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.131596 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.133664 (+ 2068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47756: BlockingRecv error: recv got EOF from 127.0.0.1:47756 (error 108)
Metrics: {"server-negotiator.queue_time_us":283}
W20260619 15:02:19.134668 2056 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.113253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43434 (local address 127.11.175.253:35165)
0619 15:02:19.113506 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.113515 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.133072 (+ 19557us) server_negotiation.cc:407] Connection header received
0619 15:02:19.133345 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.133351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.133433 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.133503 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.134421 (+ 918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43434: BlockingRecv error: recv got EOF from 127.0.0.1:43434 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 15:02:19.148731 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:19.140002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43436 (local address 127.11.175.253:35165)
0619 15:02:19.140206 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.140229 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.141114 (+ 885us) server_negotiation.cc:407] Connection header received
0619 15:02:19.141363 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.141370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.141607 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.141791 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:19.142249 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.143549 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.145624 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.146442 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.146590 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:19.147243 (+ 653us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:19.147987 (+ 744us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:19.148177 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:19.148238 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 15:02:19.148335 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260619 15:02:19.160373 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.157687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47766 (local address 127.11.175.252:42011)
0619 15:02:19.157933 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.157941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.158487 (+ 546us) server_negotiation.cc:407] Connection header received
0619 15:02:19.158886 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.158893 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.159046 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.159170 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.160108 (+ 938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47766: BlockingRecv error: recv got EOF from 127.0.0.1:47766 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
I20260619 15:02:19.177248 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:19.169813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43440 (local address 127.11.175.253:35165)
0619 15:02:19.170004 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.170011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.170537 (+ 526us) server_negotiation.cc:407] Connection header received
0619 15:02:19.170777 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.170784 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.170882 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.170994 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:19.171491 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.172834 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.174916 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.175697 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.175838 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:19.176323 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:19.176493 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:19.176594 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:19.176763 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 15:02:19.176873 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260619 15:02:19.191186 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.188762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.11.175.252:42011)
0619 15:02:19.188952 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.188958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.189557 (+ 599us) server_negotiation.cc:407] Connection header received
0619 15:02:19.189760 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.189767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.189885 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.190005 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.190914 (+ 909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47776: BlockingRecv error: recv got EOF from 127.0.0.1:47776 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 15:02:19.207868 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:19.199829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43444 (local address 127.11.175.253:35165)
0619 15:02:19.200022 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.200029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.200550 (+ 521us) server_negotiation.cc:407] Connection header received
0619 15:02:19.200822 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.200829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.200925 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.201037 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:19.201524 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.202882 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.205346 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.206105 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.206248 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:19.206887 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:19.207044 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:19.207149 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:19.207372 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 15:02:19.207484 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260619 15:02:19.243827 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.241549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.11.175.252:42011)
0619 15:02:19.241760 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.241767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.241961 (+ 194us) server_negotiation.cc:407] Connection header received
0619 15:02:19.242309 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.242316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.242418 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.242493 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.243570 (+ 1077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47784: BlockingRecv error: recv got EOF from 127.0.0.1:47784 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 15:02:19.262398 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:19.254830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43460 (local address 127.11.175.253:35165)
0619 15:02:19.255068 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.255075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.255624 (+ 549us) server_negotiation.cc:407] Connection header received
0619 15:02:19.255930 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.255937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.256033 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.256097 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:19.256781 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.257999 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.260195 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.260928 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.261065 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:19.261593 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:19.261741 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:19.261835 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:19.261999 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 15:02:19.262099 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
W20260619 15:02:19.272778 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.270301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47788 (local address 127.11.175.252:42011)
0619 15:02:19.270556 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.270562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.271093 (+ 531us) server_negotiation.cc:407] Connection header received
0619 15:02:19.271378 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.271384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.271476 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.271546 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.272552 (+ 1006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47788: BlockingRecv error: recv got EOF from 127.0.0.1:47788 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
I20260619 15:02:19.293565 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:19.285549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.11.175.253:35165)
0619 15:02:19.285745 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.285752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.286314 (+ 562us) server_negotiation.cc:407] Connection header received
0619 15:02:19.286680 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.286687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.286784 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.286860 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:19.287585 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.288784 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.290890 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.291692 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.291802 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:19.292535 (+ 733us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:19.292751 (+ 216us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:19.292828 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:19.293096 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 15:02:19.293197 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260619 15:02:19.307367 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.304643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.11.175.252:42011)
0619 15:02:19.304826 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.304833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.305388 (+ 555us) server_negotiation.cc:407] Connection header received
0619 15:02:19.305688 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.305695 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.305797 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.305875 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.307064 (+ 1189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47796: BlockingRecv error: recv got EOF from 127.0.0.1:47796 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 15:02:19.335812 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:19.328198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43476 (local address 127.11.175.253:35165)
0619 15:02:19.328409 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.328417 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.329249 (+ 832us) server_negotiation.cc:407] Connection header received
0619 15:02:19.329498 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.329505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.329601 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.329665 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:19.330320 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.331548 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.333509 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.334203 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.334382 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:19.334952 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:19.335129 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:19.335223 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:19.335408 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 15:02:19.335496 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260619 15:02:19.348506 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.346152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47802 (local address 127.11.175.252:42011)
0619 15:02:19.346348 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.346354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.346948 (+ 594us) server_negotiation.cc:407] Connection header received
0619 15:02:19.347161 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.347168 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.347266 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.347394 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.348267 (+ 873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47802: BlockingRecv error: recv got EOF from 127.0.0.1:47802 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 15:02:19.391899 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:19.384538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.11.175.253:35165)
0619 15:02:19.384756 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.384762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.385287 (+ 525us) server_negotiation.cc:407] Connection header received
0619 15:02:19.385542 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.385548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.385654 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.385777 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:19.386312 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.387489 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.389514 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.390263 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.390405 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:19.390998 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:19.391153 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:19.391256 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:19.391441 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 15:02:19.391529 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260619 15:02:19.402169 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.399168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47810 (local address 127.11.175.252:42011)
0619 15:02:19.399399 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.399405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.400464 (+ 1059us) server_negotiation.cc:407] Connection header received
0619 15:02:19.400728 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.400734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.400871 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.400995 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.401905 (+ 910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47810: BlockingRecv error: recv got EOF from 127.0.0.1:47810 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 15:02:19.475800 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:19.469702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.11.175.253:35165)
0619 15:02:19.469900 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.469906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.470268 (+ 362us) server_negotiation.cc:407] Connection header received
0619 15:02:19.470440 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.470446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.470543 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.470708 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:19.471110 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.472370 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.473779 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.474465 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.474619 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:19.475028 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:19.475175 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:19.475271 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:19.475403 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 15:02:19.475493 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260619 15:02:19.485347 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.483022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47816 (local address 127.11.175.252:42011)
0619 15:02:19.483215 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.483222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.483729 (+ 507us) server_negotiation.cc:407] Connection header received
0619 15:02:19.483894 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.483898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.483990 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.484079 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.485085 (+ 1006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47816: BlockingRecv error: recv got EOF from 127.0.0.1:47816 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 15:02:19.622711 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:19.617105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.11.175.253:35165)
0619 15:02:19.617371 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.617381 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.617658 (+ 277us) server_negotiation.cc:407] Connection header received
0619 15:02:19.617901 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.617912 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.617979 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.618082 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:19.618549 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.619268 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.620636 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.621089 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.621221 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:19.621706 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:19.621922 (+ 216us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:19.622057 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:19.622248 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 15:02:19.622336 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260619 15:02:19.630604 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.628638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.11.175.252:42011)
0619 15:02:19.628878 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.628886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.629167 (+ 281us) server_negotiation.cc:407] Connection header received
0619 15:02:19.629395 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.629401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.629488 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.629598 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.630365 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47830: BlockingRecv error: recv got EOF from 127.0.0.1:47830 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 15:02:19.896065 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:19.890825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43510 (local address 127.11.175.253:35165)
0619 15:02:19.891072 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.891077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.891270 (+ 193us) server_negotiation.cc:407] Connection header received
0619 15:02:19.891429 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.891433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.891503 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.891590 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:19.892043 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.892901 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.894347 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:19.894846 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:19.894954 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:19.895390 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:19.895492 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:19.895561 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:19.895705 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 15:02:19.895770 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 15:02:19.903354 2165 catalog_manager.cc:1259] Loaded cluster ID: f32eb84812f54d57a0d608ffa9a365f4
I20260619 15:02:19.903515 2165 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601: loading cluster ID for follower catalog manager: success
I20260619 15:02:19.907908 2165 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601: acquiring CA information for follower catalog manager: success
W20260619 15:02:19.909070 2132 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:19.904766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47844 (local address 127.11.175.252:42011)
0619 15:02:19.905008 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:19.905015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:19.907271 (+ 2256us) server_negotiation.cc:407] Connection header received
0619 15:02:19.907502 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:19.907507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:19.907901 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:19.908033 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:19.908789 (+ 756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47844: BlockingRecv error: recv got EOF from 127.0.0.1:47844 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 15:02:19.911383 2165 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f5d50a993a164f1d832d28ca9f9c1601: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 15:02:20.430459 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:20.424854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43518 (local address 127.11.175.253:35165)
0619 15:02:20.425399 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:20.425406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:20.425739 (+ 333us) server_negotiation.cc:407] Connection header received
0619 15:02:20.425821 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:20.425825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:20.425894 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:20.425990 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:20.426440 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:20.427251 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:20.428801 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:20.429187 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:20.429280 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:20.429715 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:20.429817 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:20.429887 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:20.430025 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 15:02:20.430089 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":253,"threads_started":1}
I20260619 15:02:20.442390 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:20.436548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47860 (local address 127.11.175.252:42011)
0619 15:02:20.437059 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:20.437065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:20.437085 (+ 20us) server_negotiation.cc:407] Connection header received
0619 15:02:20.437260 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:20.437265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:20.437432 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:20.437562 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:20.437973 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:20.438753 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:20.440146 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:20.440559 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:20.440700 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:20.441219 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:20.441772 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:20.441937 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:20.442004 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 15:02:20.442095 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":268,"threads_started":1}
I20260619 15:02:20.463037 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:20.449324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43532 (local address 127.11.175.253:35165)
0619 15:02:20.449486 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:20.449493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:20.456268 (+ 6775us) server_negotiation.cc:407] Connection header received
0619 15:02:20.456522 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:20.456532 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:20.456657 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:20.456774 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:20.457558 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:20.458699 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:20.460882 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:20.461528 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:20.461647 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:20.462261 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:20.462379 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:20.462448 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:20.462701 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 15:02:20.462781 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 15:02:20.464515 2321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:20.449003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.11.175.254:42085)
0619 15:02:20.457312 (+ 8309us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:20.457319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:20.457341 (+ 22us) server_negotiation.cc:407] Connection header received
0619 15:02:20.457459 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:20.457465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:20.457722 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:20.457910 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:20.458744 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:20.459817 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:20.461658 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:20.462079 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:20.462213 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:20.463070 (+ 857us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:20.463528 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:20.463633 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:20.464048 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 15:02:20.464118 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8169,"thread_start_us":8066,"threads_started":1}
I20260619 15:02:20.464735 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:20.449504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47868 (local address 127.11.175.252:42011)
0619 15:02:20.449696 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:20.449705 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:20.458620 (+ 8915us) server_negotiation.cc:407] Connection header received
0619 15:02:20.458825 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:20.458834 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:20.458948 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:20.459034 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:20.459706 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:20.460652 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:20.462692 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:20.463289 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:20.463399 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:20.463971 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:20.464095 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:20.464187 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:20.464396 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 15:02:20.464480 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 15:02:20.470420 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1910
I20260619 15:02:20.471612 1972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:02:20.584501 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 1910
I20260619 15:02:20.605561 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 1984
I20260619 15:02:20.606871 2046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:02:20.717931 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 1984
I20260619 15:02:20.738593 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 2059
I20260619 15:02:20.739794 2121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:02:20.847889 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 2059
2026-06-19T15:02:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5083 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 15:02:20 dist-test-slave-76rk krb5kdc[2330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 15:02:20 dist-test-slave-76rk krb5kdc[2330](info): set up 2 sockets
Jun 19 15:02:20 dist-test-slave-76rk krb5kdc[2330](info): commencing operation
krb5kdc: starting...
W20260619 15:02:22.003275 11967 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/test-user.keytab.
Jun 19 15:02:22 dist-test-slave-76rk krb5kdc[2330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T15:02:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T15:02:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.175.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
Entry for principal kudu/127.11.175.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab.
I20260619 15:02:22.292496 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41529
--webserver_interface=127.11.175.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:41529,127.11.175.253:37153,127.11.175.252:41039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 15:02:22.657569 2343 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 15:02:22.657969 2343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:22.658088 2343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:22.658233 2343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:22.665872 2343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:22.665982 2343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:22.666075 2343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:22.666155 2343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:22.666275 2343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:22.676880 2343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:41529,127.11.175.253:37153,127.11.175.252:41039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.254.keytab
--principal=kudu/127.11.175.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.254:41529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.175.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:22.678391 2343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:22.679932 2343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:22.692741 2348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:22.692741 2351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:22.693061 2349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:22.694124 2343 server_base.cc:1034] running on GCE node
I20260619 15:02:22.695363 2343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:22.698125 2343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:22.699436 2343 hybrid_clock.cc:648] HybridClock initialized: now 1781881342699346 us; error 63 us; skew 500 ppm
I20260619 15:02:22.699867 2343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:22 dist-test-slave-76rk krb5kdc[2330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881342, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:22.703279 2343 init.cc:377] Logged in from keytab as kudu/127.11.175.254@KRBTEST.COM (short username kudu)
I20260619 15:02:22.704881 2343 webserver.cc:460] Webserver started at http://127.11.175.254:39311/ using document root <none> and password file <none>
I20260619 15:02:22.705637 2343 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:22.705782 2343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:22.706238 2343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:22.709539 2343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/data/instance:
uuid: "e08a5d4d130449248a60b502f5f604cf"
format_stamp: "Formatted at 2026-06-19 15:02:22 on dist-test-slave-76rk"
server_key: "8469cd975349e30187ca3f1c1b50c2e4"
server_key_iv: "2dd18a5ce43598d908c7fea6a0b1dbea"
server_key_version: "encryptionkey@0"
I20260619 15:02:22.710376 2343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/wal/instance:
uuid: "e08a5d4d130449248a60b502f5f604cf"
format_stamp: "Formatted at 2026-06-19 15:02:22 on dist-test-slave-76rk"
server_key: "8469cd975349e30187ca3f1c1b50c2e4"
server_key_iv: "2dd18a5ce43598d908c7fea6a0b1dbea"
server_key_version: "encryptionkey@0"
I20260619 15:02:22.715960 2343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 15:02:22.720454 2358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:22.722041 2343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 15:02:22.722308 2343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/wal
uuid: "e08a5d4d130449248a60b502f5f604cf"
format_stamp: "Formatted at 2026-06-19 15:02:22 on dist-test-slave-76rk"
server_key: "8469cd975349e30187ca3f1c1b50c2e4"
server_key_iv: "2dd18a5ce43598d908c7fea6a0b1dbea"
server_key_version: "encryptionkey@0"
I20260619 15:02:22.722564 2343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:22.754364 2343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:22.758978 2343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:22.759287 2343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:22.779878 2343 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.254:41529
I20260619 15:02:22.779901 2409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.254:41529 every 8 connection(s)
I20260619 15:02:22.781251 2343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/data/info.pb
I20260619 15:02:22.781826 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 2343
I20260619 15:02:22.782058 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-0/wal/instance
I20260619 15:02:22.782720 11967 external_mini_cluster.cc:1428] Setting key ae43e7bd7963c92bade01536317ae8ce
I20260619 15:02:22.785518 2410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:22.792479 2410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 15:02:22 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881342, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.175.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.253@KRBTEST.COM" created.
I20260619 15:02:22.817950 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:22.796543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41529 (local address 127.0.0.1:34996)
0619 15:02:22.797406 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:22.797463 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:22.799564 (+ 2101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:22.800430 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:22.800437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:22.801154 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:22.801508 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:22.801526 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:22.803192 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:22.803196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:22.804300 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:22.804312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:22.805259 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:22.805266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:22.805434 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:22.806155 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:22.806185 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:22.808884 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:22.812570 (+ 3686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:22.812583 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:22.812597 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:22.813032 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:22.813475 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:22.813483 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:22.813488 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:22.813664 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:22.815043 (+ 1379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:22.815058 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:22.815342 (+ 284us) client_negotiation.cc:769] Sending connection context
0619 15:02:22.815657 (+ 315us) client_negotiation.cc:240] Negotiation successful
0619 15:02:22.816404 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":193,"thread_start_us":379,"threads_started":1}
I20260619 15:02:22.817973 2414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:22.796544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34996 (local address 127.11.175.254:41529)
0619 15:02:22.798223 (+ 1679us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:22.798236 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:22.798289 (+ 53us) server_negotiation.cc:407] Connection header received
0619 15:02:22.799949 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:22.799955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:22.800204 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:22.800408 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:22.801732 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:22.803019 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:22.804439 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:22.805012 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:22.805233 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:22.809140 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:22.809148 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:22.809161 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:22.809192 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:22.812398 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:22.813187 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:22.813202 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:22.813205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:22.813358 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:22.813773 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:22.813781 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:22.813784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:22.814614 (+ 830us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:22.815150 (+ 536us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:22.815798 (+ 648us) server_negotiation.cc:299] Negotiation successful
0619 15:02:22.816263 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":1452,"thread_start_us":1242,"threads_started":1}
I20260619 15:02:22.821753 2410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } has no permanent_uuid. Determining permanent_uuid...
I20260619 15:02:22.823172 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:22.822606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41428)
0619 15:02:22.822859 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:22.823001 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260619 15:02:22.823506 2361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
I20260619 15:02:22.825403 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:22.824895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41440)
0619 15:02:22.825094 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:22.825192 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260619 15:02:22.825729 2410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:37153: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab.
Entry for principal kudu/127.11.175.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab.
I20260619 15:02:22.846925 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.253:37153
--webserver_interface=127.11.175.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab
--principal=kudu/127.11.175.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:41529,127.11.175.253:37153,127.11.175.252:41039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 15:02:22.875154 2410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } attempt: 1
I20260619 15:02:22.876571 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:22.875917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41456)
0619 15:02:22.876190 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:22.876376 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260619 15:02:22.877743 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:22.877319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41460)
0619 15:02:22.877482 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:22.877578 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 15:02:22.877993 2410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:37153: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
I20260619 15:02:22.946308 2410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } attempt: 2
I20260619 15:02:22.947680 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:22.947155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41476)
0619 15:02:22.947433 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:22.947526 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260619 15:02:22.948786 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:22.948411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41478)
0619 15:02:22.948573 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:22.948650 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260619 15:02:22.949038 2410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:37153: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
I20260619 15:02:23.040406 2410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } attempt: 3
I20260619 15:02:23.041731 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.041240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41490)
0619 15:02:23.041496 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.041591 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260619 15:02:23.043020 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.042547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41492)
0619 15:02:23.042778 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.042857 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260619 15:02:23.043218 2410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:37153: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
I20260619 15:02:23.186450 2410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } attempt: 4
I20260619 15:02:23.187733 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.187124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41502)
0619 15:02:23.187485 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.187586 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260619 15:02:23.188771 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.188408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41510)
0619 15:02:23.188556 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.188643 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 15:02:23.188948 2410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.253:37153: Network error: Client connection negotiation failed: client connection to 127.11.175.253:37153: connect: Connection refused (error 111)
W20260619 15:02:23.197752 2417 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 15:02:23.198167 2417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:23.198308 2417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:23.198453 2417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:23.207278 2417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:23.207417 2417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:23.207526 2417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:23.207614 2417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:23.207705 2417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:23.218820 2417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:41529,127.11.175.253:37153,127.11.175.252:41039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.253.keytab
--principal=kudu/127.11.175.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.253:37153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.175.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:23.220261 2417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:23.221701 2417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:23.235747 2425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:23.235967 2422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:23.236464 2423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:23.237162 2417 server_base.cc:1034] running on GCE node
I20260619 15:02:23.238085 2417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:23.239835 2417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:23.241123 2417 hybrid_clock.cc:648] HybridClock initialized: now 1781881343241039 us; error 55 us; skew 500 ppm
I20260619 15:02:23.241617 2417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:23.245399 2417 init.cc:377] Logged in from keytab as kudu/127.11.175.253@KRBTEST.COM (short username kudu)
I20260619 15:02:23.247422 2417 webserver.cc:460] Webserver started at http://127.11.175.253:43587/ using document root <none> and password file <none>
I20260619 15:02:23.248265 2417 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:23.248423 2417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:23.248805 2417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:23.252313 2417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/data/instance:
uuid: "2a2f14113e3d47e39280e7308b999239"
format_stamp: "Formatted at 2026-06-19 15:02:23 on dist-test-slave-76rk"
server_key: "5d7952d02e2d8f6614c33399628f4b75"
server_key_iv: "1887e600262a887de8a02fcb26f9b93d"
server_key_version: "encryptionkey@0"
I20260619 15:02:23.253361 2417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/wal/instance:
uuid: "2a2f14113e3d47e39280e7308b999239"
format_stamp: "Formatted at 2026-06-19 15:02:23 on dist-test-slave-76rk"
server_key: "5d7952d02e2d8f6614c33399628f4b75"
server_key_iv: "1887e600262a887de8a02fcb26f9b93d"
server_key_version: "encryptionkey@0"
I20260619 15:02:23.259702 2417 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260619 15:02:23.263935 2432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:23.265797 2417 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 15:02:23.266048 2417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/wal
uuid: "2a2f14113e3d47e39280e7308b999239"
format_stamp: "Formatted at 2026-06-19 15:02:23 on dist-test-slave-76rk"
server_key: "5d7952d02e2d8f6614c33399628f4b75"
server_key_iv: "1887e600262a887de8a02fcb26f9b93d"
server_key_version: "encryptionkey@0"
I20260619 15:02:23.266343 2417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:23.280387 2417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:23.285382 2417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:23.285682 2417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:23.306938 2417 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.253:37153
I20260619 15:02:23.306988 2483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.253:37153 every 8 connection(s)
I20260619 15:02:23.308495 2417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/data/info.pb
I20260619 15:02:23.312599 2484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:23.318578 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 2417
I20260619 15:02:23.318816 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-1/wal/instance
I20260619 15:02:23.319339 11967 external_mini_cluster.cc:1428] Setting key 775378fa0407a54c3ee919b348a5615f
I20260619 15:02:23.320883 2484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:02:23.346673 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.324678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.11.175.254:41529)
0619 15:02:23.325227 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:23.325235 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:23.326474 (+ 1239us) server_negotiation.cc:407] Connection header received
0619 15:02:23.328572 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:23.328580 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:23.328685 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:23.328818 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:23.330457 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.331349 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.333980 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.334444 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.334555 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.339392 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:23.339399 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:23.339402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:23.339463 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:23.342723 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.343755 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.343764 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.343770 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.343943 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.344449 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.344458 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.344463 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.344932 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:23.345126 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:23.345910 (+ 784us) server_negotiation.cc:299] Negotiation successful
0619 15:02:23.346172 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":258,"threads_started":1}
I20260619 15:02:23.347617 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.325353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41529 (local address 127.0.0.1:35004)
0619 15:02:23.326283 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.326342 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:23.328267 (+ 1925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:23.329041 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:23.329050 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:23.329623 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:23.330206 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.330225 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.331539 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.331548 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.333785 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.333807 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.334742 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.334752 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.335093 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.335948 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:23.335991 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:23.339095 (+ 3104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:23.342911 (+ 3816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.342923 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.342943 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.343563 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.344111 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.344116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.344119 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.344313 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.345102 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:23.345112 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:23.345499 (+ 387us) client_negotiation.cc:769] Sending connection context
0619 15:02:23.345770 (+ 271us) client_negotiation.cc:240] Negotiation successful
0619 15:02:23.346218 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":409,"threads_started":1}
WARNING: no policy specified for kudu/127.11.175.252@KRBTEST.COM; defaulting to no policy
I20260619 15:02:23.349642 2484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.175.252@KRBTEST.COM" created.
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 15:02:23.367463 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.350543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41516)
0619 15:02:23.350846 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.350864 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:23.350994 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:23.351922 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:23.351930 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:23.352462 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:23.352702 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.352717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.354083 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.354089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.355613 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.355633 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.356526 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.356534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.356770 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.357477 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:23.357515 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:23.360323 (+ 2808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:23.364606 (+ 4283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.364610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.364613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.365014 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.365482 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.365486 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.365488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.365670 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.366457 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:23.366461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:23.366586 (+ 125us) client_negotiation.cc:769] Sending connection context
0619 15:02:23.366965 (+ 379us) client_negotiation.cc:240] Negotiation successful
0619 15:02:23.367186 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260619 15:02:23.368005 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.350781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41516 (local address 127.11.175.253:37153)
0619 15:02:23.351477 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:23.351483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:23.351508 (+ 25us) server_negotiation.cc:407] Connection header received
0619 15:02:23.351551 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:23.351556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:23.351746 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:23.351862 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:23.352874 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.353936 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.355776 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.356355 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.356463 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.360547 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:23.360561 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:23.360575 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:23.360637 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:23.364422 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.365176 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.365188 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.365194 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.365339 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.365804 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.365812 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.365818 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.366316 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:23.366521 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:23.367182 (+ 661us) server_negotiation.cc:299] Negotiation successful
0619 15:02:23.367509 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":405,"threads_started":1}
I20260619 15:02:23.370498 2484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } has no permanent_uuid. Determining permanent_uuid...
I20260619 15:02:23.371866 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.371360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37008)
0619 15:02:23.371552 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.371712 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 15:02:23.372172 2436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
I20260619 15:02:23.373848 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.373478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37018)
0619 15:02:23.373657 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.373736 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 15:02:23.374089 2484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:41039: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.175.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab.
Entry for principal kudu/127.11.175.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab.
I20260619 15:02:23.390918 11967 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
/tmp/dist-test-taskDoOfXv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.252:41039
--webserver_interface=127.11.175.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab
--principal=kudu/127.11.175.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.175.212:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.175.254:41529,127.11.175.253:37153,127.11.175.252:41039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 15:02:23.423413 2484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } attempt: 1
I20260619 15:02:23.424777 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.424190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37028)
0619 15:02:23.424437 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.424568 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260619 15:02:23.425990 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.425571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37038)
0619 15:02:23.425744 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.425841 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 15:02:23.426324 2484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:41039: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
I20260619 15:02:23.488204 2410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } attempt: 5
I20260619 15:02:23.494549 2484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } attempt: 2
I20260619 15:02:23.495546 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.495116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37050)
0619 15:02:23.495309 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.495398 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 15:02:23.496726 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.496331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37056)
0619 15:02:23.496517 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.496600 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260619 15:02:23.497010 2484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:41039: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881342, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 15:02:23.504041 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.489130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41520 (local address 127.11.175.253:37153)
0619 15:02:23.489294 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:23.489300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:23.489406 (+ 106us) server_negotiation.cc:407] Connection header received
0619 15:02:23.489575 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:23.489580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:23.489708 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:23.489842 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:23.490825 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.492018 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.493285 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.493936 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.494066 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.498251 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:23.498256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:23.498260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:23.498294 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:23.501110 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.501951 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.501956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.501958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.502133 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.502586 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.502590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.502592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.503047 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:23.503205 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:23.503662 (+ 457us) server_negotiation.cc:299] Negotiation successful
0619 15:02:23.503830 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 15:02:23.504218 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.489000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41520)
0619 15:02:23.489231 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.489254 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:23.489445 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:23.489901 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:23.489905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:23.490366 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:23.490553 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.490562 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.492213 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.492226 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.493115 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.493129 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.494157 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.494162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.494344 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.495094 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:23.495120 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:23.498012 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:23.501275 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.501280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.501285 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.501815 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.502252 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.502257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.502261 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.502440 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.503210 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:23.503217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:23.503389 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 15:02:23.503560 (+ 171us) client_negotiation.cc:240] Negotiation successful
0619 15:02:23.503888 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260619 15:02:23.505263 2410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } has no permanent_uuid. Determining permanent_uuid...
I20260619 15:02:23.506531 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.506032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37058)
0619 15:02:23.506274 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.506372 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260619 15:02:23.507576 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.507212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37066)
0619 15:02:23.507357 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.507453 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 15:02:23.507862 2410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:41039: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
I20260619 15:02:23.559150 2410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } attempt: 1
I20260619 15:02:23.560652 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.560063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37076)
0619 15:02:23.560325 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.560439 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260619 15:02:23.561797 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.561404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37080)
0619 15:02:23.561564 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.561653 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 15:02:23.562052 2410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:41039: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
I20260619 15:02:23.588266 2484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } attempt: 3
I20260619 15:02:23.589624 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.588944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37090)
0619 15:02:23.589229 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.589368 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 15:02:23.591022 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.590399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37096)
0619 15:02:23.590723 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.590849 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260619 15:02:23.591292 2484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:41039: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
I20260619 15:02:23.630303 2410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } attempt: 2
I20260619 15:02:23.631630 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.631053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37102)
0619 15:02:23.631292 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.631405 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 15:02:23.632845 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.632401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37112)
0619 15:02:23.632601 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.632701 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 15:02:23.633082 2410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:41039: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
I20260619 15:02:23.734562 2484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } attempt: 4
I20260619 15:02:23.736003 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.735395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37122)
0619 15:02:23.735677 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.735795 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260619 15:02:23.737393 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.736970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37134)
0619 15:02:23.737152 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.737246 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260619 15:02:23.737674 2484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:41039: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
I20260619 15:02:23.739305 2410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } attempt: 3
I20260619 15:02:23.740394 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.739922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37150)
0619 15:02:23.740122 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.740220 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 15:02:23.741417 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.741011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37158)
0619 15:02:23.741212 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.741297 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260619 15:02:23.741647 2410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.175.252:41039: Network error: Client connection negotiation failed: client connection to 127.11.175.252:41039: connect: Connection refused (error 111)
W20260619 15:02:23.752555 2492 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 15:02:23.752995 2492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 15:02:23.753157 2492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 15:02:23.753340 2492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 15:02:23.762244 2492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 15:02:23.762362 2492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 15:02:23.762470 2492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 15:02:23.762553 2492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 15:02:23.762686 2492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 15:02:23.773145 2492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.175.212:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.175.254:41529,127.11.175.253:37153,127.11.175.252:41039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/krb5kdc/kudu_127.11.175.252.keytab
--principal=kudu/127.11.175.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.175.252:41039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.175.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 13:58:13 UTC on ccycloud.quasar-psecfj.root.comops.site
build id 4455767
ASAN enabled
I20260619 15:02:23.774528 2492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 15:02:23.775923 2492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 15:02:23.789206 2498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:23.789299 2500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 15:02:23.789901 2497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 15:02:23.792248 2492 server_base.cc:1034] running on GCE node
I20260619 15:02:23.793115 2492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 15:02:23.794693 2492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 15:02:23.795957 2492 hybrid_clock.cc:648] HybridClock initialized: now 1781881343795870 us; error 63 us; skew 500 ppm
I20260619 15:02:23.796314 2492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 15:02:23.809016 2492 init.cc:377] Logged in from keytab as kudu/127.11.175.252@KRBTEST.COM (short username kudu)
I20260619 15:02:23.810958 2492 webserver.cc:460] Webserver started at http://127.11.175.252:34133/ using document root <none> and password file <none>
I20260619 15:02:23.811774 2492 fs_manager.cc:359] Metadata directory not provided
I20260619 15:02:23.811919 2492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 15:02:23.812253 2492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 15:02:23.815480 2492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/data/instance:
uuid: "f03fd8ef3e8340fea14a7e0aca59953f"
format_stamp: "Formatted at 2026-06-19 15:02:23 on dist-test-slave-76rk"
server_key: "869a181f5b0779a27681132ef89a4298"
server_key_iv: "6a94f0148170af4ff6e4d1a800ff6ce5"
server_key_version: "encryptionkey@0"
I20260619 15:02:23.816320 2492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/wal/instance:
uuid: "f03fd8ef3e8340fea14a7e0aca59953f"
format_stamp: "Formatted at 2026-06-19 15:02:23 on dist-test-slave-76rk"
server_key: "869a181f5b0779a27681132ef89a4298"
server_key_iv: "6a94f0148170af4ff6e4d1a800ff6ce5"
server_key_version: "encryptionkey@0"
I20260619 15:02:23.822019 2492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 15:02:23.825730 2507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 15:02:23.827538 2492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 15:02:23.827767 2492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/data,/tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/wal
uuid: "f03fd8ef3e8340fea14a7e0aca59953f"
format_stamp: "Formatted at 2026-06-19 15:02:23 on dist-test-slave-76rk"
server_key: "869a181f5b0779a27681132ef89a4298"
server_key_iv: "6a94f0148170af4ff6e4d1a800ff6ce5"
server_key_version: "encryptionkey@0"
I20260619 15:02:23.828029 2492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 15:02:23.841696 2492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 15:02:23.846132 2492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 15:02:23.846436 2492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 15:02:23.865953 2492 rpc_server.cc:307] RPC server started. Bound to: 127.11.175.252:41039
I20260619 15:02:23.865978 2558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.175.252:41039 every 8 connection(s)
I20260619 15:02:23.867504 2492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/data/info.pb
I20260619 15:02:23.870164 11967 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu as pid 2492
I20260619 15:02:23.870374 11967 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDoOfXv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781880972997616-11967-0/minicluster-data/master-2/wal/instance
I20260619 15:02:23.871156 11967 external_mini_cluster.cc:1428] Setting key acb03235712d53885cab3904d2b068b2
I20260619 15:02:23.872356 2559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 15:02:23.882627 2559 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
I20260619 15:02:23.897185 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.873261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.11.175.252:41039)
0619 15:02:23.874186 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:23.874199 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:23.874245 (+ 46us) server_negotiation.cc:407] Connection header received
0619 15:02:23.876041 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:23.876067 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:23.876558 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:23.876982 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:23.878949 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.880413 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.882175 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.883030 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.883190 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.888254 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:23.888265 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:23.888280 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:23.888332 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:23.892214 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.893104 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.893113 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.893122 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.893344 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.893800 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.893806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.893809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.894412 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:23.894794 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:23.895273 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 15:02:23.895844 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":424,"threads_started":1}
I20260619 15:02:23.903273 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.886015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.254:41529 (local address 127.0.0.1:35020)
0619 15:02:23.887013 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.887062 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:23.887232 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:23.887666 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:23.887671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:23.888543 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:23.888925 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.888939 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.890075 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.890079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.891935 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.891953 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.893084 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.893092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.893344 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.894018 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:23.894041 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:23.896512 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:23.900168 (+ 3656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.900180 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.900195 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.900870 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.901349 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.901357 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.901360 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.901552 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.902296 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:23.902302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:23.902446 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 15:02:23.902596 (+ 150us) client_negotiation.cc:240] Negotiation successful
0619 15:02:23.902913 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":564,"threads_started":1}
I20260619 15:02:23.903535 2563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.886004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.11.175.254:41529)
0619 15:02:23.886585 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:23.886591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:23.887208 (+ 617us) server_negotiation.cc:407] Connection header received
0619 15:02:23.887373 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:23.887379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:23.887470 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:23.887596 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:23.889090 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.889902 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.892165 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.892853 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.893005 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.896721 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:23.896726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:23.896729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:23.896780 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:23.900012 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.901046 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.901050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.901053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.901213 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.901727 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.901734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.901737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.902148 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:23.902327 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:23.902831 (+ 504us) server_negotiation.cc:299] Negotiation successful
0619 15:02:23.903065 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":292,"threads_started":1}
I20260619 15:02:23.904914 2559 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.254@KRBTEST.COM
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 15:02:23.918324 2565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.900549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.11.175.254:41529)
0619 15:02:23.901147 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:23.901152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:23.901170 (+ 18us) server_negotiation.cc:407] Connection header received
0619 15:02:23.901209 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:23.901212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:23.901296 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:23.901412 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:23.903191 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.904426 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.905934 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.906616 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.906803 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.911034 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:23.911039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:23.911042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:23.911074 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:23.914439 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.915828 (+ 1389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.915833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.915835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.915991 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.916494 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.916499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.916502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.916847 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:23.917126 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:23.917576 (+ 450us) server_negotiation.cc:299] Negotiation successful
0619 15:02:23.917859 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":286,"threads_started":1}
I20260619 15:02:23.918358 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.905690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.253:37153 (local address 127.0.0.1:41534)
0619 15:02:23.905854 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.905872 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:23.905964 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:23.906237 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:23.906240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:23.906605 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:23.906811 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.906820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.907744 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.907752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.908615 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.908625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.909405 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.909411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.909600 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.910147 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:23.910164 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:23.912469 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:23.915814 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.915820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.915824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.916227 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.916654 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.916659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.916664 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.916832 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.917368 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:23.917373 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:23.917545 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 15:02:23.917689 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 15:02:23.917990 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 15:02:23.918391 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.905743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.11.175.253:37153)
0619 15:02:23.905951 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:23.905957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:23.905975 (+ 18us) server_negotiation.cc:407] Connection header received
0619 15:02:23.906040 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:23.906045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:23.906120 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:23.906216 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:23.906956 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.907606 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.908773 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.909250 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.909377 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.912704 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:23.912709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:23.912713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:23.912748 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:23.915644 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.916376 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.916382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.916391 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.916531 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.916950 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.916956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.916960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.917232 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:23.917400 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:23.917793 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 15:02:23.918017 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 15:02:23.918874 2410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } attempt: 4
I20260619 15:02:23.919947 2559 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881342, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.254@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.252@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
Jun 19 15:02:23 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.175.253@KRBTEST.COM
I20260619 15:02:23.935895 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.919493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37174)
0619 15:02:23.919770 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.919792 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:23.919922 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:23.920671 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:23.920676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:23.921204 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:23.921460 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.921471 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.922881 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.922887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.924315 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.924332 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.925396 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.925403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.925595 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.926320 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:23.926347 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:23.929065 (+ 2718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:23.933018 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.933024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.933029 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.933453 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.933913 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.933917 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.933921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.934062 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.934796 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:23.934801 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:23.934961 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 15:02:23.935118 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 15:02:23.935440 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260619 15:02:23.936300 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.919646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.11.175.252:41039)
0619 15:02:23.920145 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:23.920150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:23.920184 (+ 34us) server_negotiation.cc:407] Connection header received
0619 15:02:23.920247 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:23.920252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:23.920334 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:23.920433 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:23.921614 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.922709 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.924468 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.925221 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.925345 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.929271 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:23.929277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:23.929282 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:23.929321 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:23.932857 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.933584 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.933592 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.933599 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.933779 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.934175 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.934180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.934183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.934554 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:23.934799 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:23.935231 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 15:02:23.935545 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":368}
I20260619 15:02:23.938787 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.920701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37182)
0619 15:02:23.920958 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:23.920981 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:23.921097 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:23.922829 (+ 1732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:23.922833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:23.923362 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:23.923542 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.923555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.924935 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.924940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.926217 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:23.926235 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.927093 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.927099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:23.927363 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.928009 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:23.928032 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:23.930952 (+ 2920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:23.935210 (+ 4258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.935214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.935220 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.935806 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.936439 (+ 633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:23.936444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:23.936447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:23.936621 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:23.937626 (+ 1005us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:23.937631 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:23.937782 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 15:02:23.937925 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 15:02:23.938333 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":26}
I20260619 15:02:23.939262 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.920685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.11.175.252:41039)
0619 15:02:23.922483 (+ 1798us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:23.922490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:23.922512 (+ 22us) server_negotiation.cc:407] Connection header received
0619 15:02:23.922558 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:23.922562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:23.922686 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:23.922802 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:23.923731 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.924789 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.926386 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.926929 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.927073 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.931200 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:23.931204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:23.931229 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:23.931278 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:23.935063 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.935943 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.935950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.936090 (+ 140us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.936239 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.936850 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.936854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.936872 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.937426 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:23.937564 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:23.938019 (+ 455us) server_negotiation.cc:299] Negotiation successful
0619 15:02:23.938459 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":225,"server-negotiator.queue_time_us":1652,"thread_start_us":1400,"threads_started":1}
I20260619 15:02:23.941706 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:23.921118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41544 (local address 127.11.175.253:37153)
0619 15:02:23.921284 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:23.921290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:23.921781 (+ 491us) server_negotiation.cc:407] Connection header received
0619 15:02:23.921956 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:23.921963 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:23.922055 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:23.922157 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:23.923920 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.924822 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.926522 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:23.927277 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:23.927444 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:23.933712 (+ 6268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:23.933719 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:23.933725 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:23.933775 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:23.938341 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.939428 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.939436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.939439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.939624 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:23.940137 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:23.940142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:23.940146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:23.940519 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:23.940796 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:23.941131 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 15:02:23.941396 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 15:02:23.942714 2410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf: Bootstrap starting.
I20260619 15:02:23.946137 2410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:23.946902 11967 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 15:02:23.947566 2410 log.cc:824] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:23.951212 2559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f: Bootstrap starting.
I20260619 15:02:23.951370 2410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf: No bootstrap required, opened a new log
I20260619 15:02:23.949781 2385 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e08a5d4d130449248a60b502f5f604cf"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35024
I20260619 15:02:23.954434 2559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:23.955615 2559 log.cc:824] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:23.959201 2559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f: No bootstrap required, opened a new log
I20260619 15:02:23.959115 2410 raft_consensus.cc:348] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } }
I20260619 15:02:23.959522 2410 raft_consensus.cc:729] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e08a5d4d130449248a60b502f5f604cf, State: Initialized, Role: FOLLOWER
I20260619 15:02:23.960041 2410 consensus_queue.cc:260] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } }
I20260619 15:02:23.961031 2569 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } } }
I20260619 15:02:23.961439 2569 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:23.962231 2410 sys_catalog.cc:564] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:23.962374 2385 raft_consensus.cc:482] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 15:02:23.962611 2385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:23.966200 2385 raft_consensus.cc:504] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } }
I20260619 15:02:23.968461 2385 leader_election.cc:288] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [CANDIDATE]: Term 1 election: Requested vote from peers 2a2f14113e3d47e39280e7308b999239 (127.11.175.253:37153), f03fd8ef3e8340fea14a7e0aca59953f (127.11.175.252:41039)
I20260619 15:02:23.970017 2559 raft_consensus.cc:348] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } }
I20260619 15:02:23.970456 2559 raft_consensus.cc:729] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f03fd8ef3e8340fea14a7e0aca59953f, State: Initialized, Role: FOLLOWER
I20260619 15:02:23.971006 2559 consensus_queue.cc:260] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } }
I20260619 15:02:23.972685 2577 sys_catalog.cc:455] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } } }
I20260619 15:02:23.973850 2559 sys_catalog.cc:564] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:23.973914 2534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e08a5d4d130449248a60b502f5f604cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f03fd8ef3e8340fea14a7e0aca59953f"
I20260619 15:02:23.974387 2534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:23.979293 2534 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e08a5d4d130449248a60b502f5f604cf in term 1.
I20260619 15:02:23.980660 2362 leader_election.cc:302] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e08a5d4d130449248a60b502f5f604cf, f03fd8ef3e8340fea14a7e0aca59953f; no voters:
I20260619 15:02:23.981305 2569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 15:02:23.973793 2577 sys_catalog.cc:458] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:23.997404 2569 raft_consensus.cc:686] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [term 1 LEADER]: Becoming Leader. State: Replica: e08a5d4d130449248a60b502f5f604cf, State: Running, Role: LEADER
I20260619 15:02:23.998301 2569 consensus_queue.cc:237] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } }
I20260619 15:02:24.008369 2580 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e08a5d4d130449248a60b502f5f604cf. Latest consensus state: current_term: 1 leader_uuid: "e08a5d4d130449248a60b502f5f604cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } } }
I20260619 15:02:24.008982 2580 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:24.009995 2597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260619 15:02:24.012645 2364 tablet.cc:2369] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:02:24.026662 2597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 15:02:24.028081 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.004565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41552 (local address 127.11.175.253:37153)
0619 15:02:24.004861 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.004870 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.004902 (+ 32us) server_negotiation.cc:407] Connection header received
0619 15:02:24.004989 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.004997 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.005105 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.005254 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.009592 (+ 4338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.010944 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.015539 (+ 4595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.016230 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.016345 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.020069 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:24.020076 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:24.020083 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:24.020138 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:24.023851 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:24.024745 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:24.024750 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:24.024753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:24.024933 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:24.025429 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:24.025434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:24.025437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:24.025988 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:24.026262 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.027400 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.027698 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260619 15:02:24.036916 2484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } attempt: 5
I20260619 15:02:24.044510 2534 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [term 1 FOLLOWER]: Refusing update from remote peer e08a5d4d130449248a60b502f5f604cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:02:24.046412 2569 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Jun 19 15:02:24 dist-test-slave-76rk krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781881343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.175.253@KRBTEST.COM for kudu/127.11.175.252@KRBTEST.COM
I20260619 15:02:24.056686 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.037783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.175.252:41039 (local address 127.0.0.1:37208)
0619 15:02:24.038282 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0619 15:02:24.038308 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 15:02:24.038470 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 15:02:24.039464 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 15:02:24.039470 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 15:02:24.040003 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0619 15:02:24.040254 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:24.040271 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.041772 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.041777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:24.043229 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 15:02:24.043244 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.044448 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.044454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 15:02:24.044704 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.045416 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 15:02:24.045446 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 15:02:24.048223 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 15:02:24.053440 (+ 5217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:24.053446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:24.053450 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:24.054012 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:24.054522 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 15:02:24.054528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 15:02:24.054531 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 15:02:24.054757 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 15:02:24.055544 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 15:02:24.055549 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 15:02:24.055752 (+ 203us) client_negotiation.cc:769] Sending connection context
0619 15:02:24.055907 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 15:02:24.056245 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328}
I20260619 15:02:24.057825 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.038776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.11.175.252:41039)
0619 15:02:24.038954 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.038959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.038978 (+ 19us) server_negotiation.cc:407] Connection header received
0619 15:02:24.039042 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.039046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.039125 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.039255 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.040445 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.041606 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.043396 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.044072 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.044189 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.048477 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:24.048484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:24.048488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:24.048528 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:24.053190 (+ 4662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:24.054193 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:24.054199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:24.054202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:24.054367 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:24.054894 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:24.054900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:24.054903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:24.055306 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:24.055473 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.057147 (+ 1674us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.057405 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 15:02:24.064352 2484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239: Bootstrap starting.
I20260619 15:02:24.068974 2484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239: Neither blocks nor log segments found. Creating new log.
I20260619 15:02:24.069485 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.006381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37194 (local address 127.11.175.252:41039)
0619 15:02:24.006765 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.006775 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.011339 (+ 4564us) server_negotiation.cc:407] Connection header received
0619 15:02:24.012116 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.012125 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.012210 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.012297 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.017501 (+ 5204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.019014 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.021701 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.022599 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.022792 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.031071 (+ 8279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:24.031082 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:24.031090 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:24.031154 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:24.041071 (+ 9917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:24.057623 (+ 16552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:24.057630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:24.057633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:24.058332 (+ 699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:24.062108 (+ 3776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:24.062114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:24.062118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:24.063004 (+ 886us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:24.063601 (+ 597us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.067952 (+ 4351us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.068526 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260619 15:02:24.070923 2484 log.cc:824] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239: Log is configured to *not* fsync() on all Append() calls
I20260619 15:02:24.080418 2484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239: No bootstrap required, opened a new log
I20260619 15:02:24.086557 2577 sys_catalog.cc:455] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e08a5d4d130449248a60b502f5f604cf. Latest consensus state: current_term: 1 leader_uuid: "e08a5d4d130449248a60b502f5f604cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } } }
I20260619 15:02:24.087056 2577 sys_catalog.cc:458] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:24.092828 2607 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260619 15:02:24.098668 2513 tablet.cc:2369] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 15:02:24.095958 2492 master_runner.cc:186] Error getting master registration for 127.11.175.254:41529: OK, instance_id { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" instance_seqno: 1781881342773614 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 15:02:24.108851 2492 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 15:02:24.117995 2484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } }
I20260619 15:02:24.118794 2484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2f14113e3d47e39280e7308b999239, State: Initialized, Role: FOLLOWER
I20260619 15:02:24.119561 2484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } }
I20260619 15:02:24.122007 2577 sys_catalog.cc:455] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e08a5d4d130449248a60b502f5f604cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } } }
I20260619 15:02:24.122368 2577 sys_catalog.cc:458] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:24.123732 2569 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e08a5d4d130449248a60b502f5f604cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } } }
I20260619 15:02:24.124125 2569 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:24.125459 2597 catalog_manager.cc:1347] Generated new cluster ID: f240833aac104a1699df0bc48ecc9928
I20260619 15:02:24.125561 2597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 15:02:24.126709 2565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.004901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.11.175.254:41529)
0619 15:02:24.005164 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.005174 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.010757 (+ 5583us) server_negotiation.cc:407] Connection header received
0619 15:02:24.011276 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.011282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.011377 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.011459 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.059763 (+ 48304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.060918 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.083382 (+ 22464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.084200 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.084372 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.114342 (+ 29970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 15:02:24.114351 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 15:02:24.114357 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 15:02:24.114405 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 15:02:24.118514 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:24.121142 (+ 2628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:24.121152 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:24.121158 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:24.121395 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 15:02:24.122576 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 15:02:24.122585 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 15:02:24.122590 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 15:02:24.123156 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 15:02:24.123447 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.125981 (+ 2534us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.126284 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 15:02:24.125384 2343 master_runner.cc:186] Error getting master registration for 127.11.175.253:37153: OK, instance_id { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" instance_seqno: 1781881343300453 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 15:02:24.142179 2343 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 15:02:24.145969 2610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } } }
I20260619 15:02:24.148718 2459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e08a5d4d130449248a60b502f5f604cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2a2f14113e3d47e39280e7308b999239"
I20260619 15:02:24.148973 2458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [term 0 FOLLOWER]: Advancing to term 1
I20260619 15:02:24.148547 2484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 15:02:24.148708 2610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:24.153340 2458 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [term 1 FOLLOWER]: Refusing update from remote peer e08a5d4d130449248a60b502f5f604cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 15:02:24.158458 2459 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e08a5d4d130449248a60b502f5f604cf in term 1.
I20260619 15:02:24.161653 2569 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 15:02:24.185999 2569 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e08a5d4d130449248a60b502f5f604cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } } }
I20260619 15:02:24.186101 2610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e08a5d4d130449248a60b502f5f604cf. Latest consensus state: current_term: 1 leader_uuid: "e08a5d4d130449248a60b502f5f604cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } } }
I20260619 15:02:24.186323 2569 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf [sys.catalog]: This master's current role is: LEADER
I20260619 15:02:24.186401 2610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:24.187755 2597 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 15:02:24.188410 2615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e08a5d4d130449248a60b502f5f604cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08a5d4d130449248a60b502f5f604cf" member_type: VOTER last_known_addr { host: "127.11.175.254" port: 41529 } } peers { permanent_uuid: "2a2f14113e3d47e39280e7308b999239" member_type: VOTER last_known_addr { host: "127.11.175.253" port: 37153 } } peers { permanent_uuid: "f03fd8ef3e8340fea14a7e0aca59953f" member_type: VOTER last_known_addr { host: "127.11.175.252" port: 41039 } } }
I20260619 15:02:24.188849 2615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 15:02:24.190296 2597 catalog_manager.cc:1504] Loading token signing keys...
I20260619 15:02:24.218356 2597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e08a5d4d130449248a60b502f5f604cf: Generated new TSK 0
I20260619 15:02:24.219548 2597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 15:02:24.226187 2626 catalog_manager.cc:1259] Loaded cluster ID: f240833aac104a1699df0bc48ecc9928
I20260619 15:02:24.226331 2626 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239: loading cluster ID for follower catalog manager: success
I20260619 15:02:24.230396 2626 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239: acquiring CA information for follower catalog manager: success
I20260619 15:02:24.233276 2626 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a2f14113e3d47e39280e7308b999239: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 15:02:24.353202 2489 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.340606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41566 (local address 127.11.175.253:37153)
0619 15:02:24.341055 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.341062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.351208 (+ 10146us) server_negotiation.cc:407] Connection header received
0619 15:02:24.351408 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.351414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.351524 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.351643 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.352940 (+ 1297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41566: BlockingRecv error: recv got EOF from 127.0.0.1:41566 (error 108)
Metrics: {"server-negotiator.queue_time_us":295}
W20260619 15:02:24.354277 2565 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.340830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.11.175.254:41529)
0619 15:02:24.341210 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.341216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.352135 (+ 10919us) server_negotiation.cc:407] Connection header received
0619 15:02:24.352403 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.352415 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.352507 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.352575 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.353997 (+ 1422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35040: BlockingRecv error: recv got EOF from 127.0.0.1:35040 (error 108)
Metrics: {"server-negotiator.queue_time_us":253}
W20260619 15:02:24.355363 2566 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.347071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37224 (local address 127.11.175.252:41039)
0619 15:02:24.347282 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.347290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.351843 (+ 4553us) server_negotiation.cc:407] Connection header received
0619 15:02:24.353739 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.353746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.353831 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.353903 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.355108 (+ 1205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37224: BlockingRecv error: recv got EOF from 127.0.0.1:37224 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 15:02:24.369045 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.360840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41576 (local address 127.11.175.253:37153)
0619 15:02:24.361031 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.361038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.361539 (+ 501us) server_negotiation.cc:407] Connection header received
0619 15:02:24.361782 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.361788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.362009 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.362194 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:24.362714 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.363945 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.366039 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.366807 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.366936 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.367573 (+ 637us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:24.368314 (+ 741us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:24.368481 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.368576 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.368685 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 15:02:24.385066 2566 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.377142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37240 (local address 127.11.175.252:41039)
0619 15:02:24.377353 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.377360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.383372 (+ 6012us) server_negotiation.cc:407] Connection header received
0619 15:02:24.383597 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.383604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.383746 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.383874 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.384811 (+ 937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37240: BlockingRecv error: recv got EOF from 127.0.0.1:37240 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 15:02:24.397555 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.389794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41584 (local address 127.11.175.253:37153)
0619 15:02:24.389998 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.390005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.390545 (+ 540us) server_negotiation.cc:407] Connection header received
0619 15:02:24.390790 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.390796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.390933 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.391055 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:24.391582 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.392786 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.395093 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.395869 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.396023 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.396559 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:24.396745 (+ 186us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:24.396928 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.397110 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.397210 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260619 15:02:24.408053 2566 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.405854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37254 (local address 127.11.175.252:41039)
0619 15:02:24.406053 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.406060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.406272 (+ 212us) server_negotiation.cc:407] Connection header received
0619 15:02:24.406602 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.406610 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.406738 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.406853 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.407799 (+ 946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37254: BlockingRecv error: recv got EOF from 127.0.0.1:37254 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 15:02:24.433995 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.417090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41592 (local address 127.11.175.253:37153)
0619 15:02:24.417308 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.417315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.426806 (+ 9491us) server_negotiation.cc:407] Connection header received
0619 15:02:24.427072 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.427079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.427235 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.427346 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:24.427893 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.429128 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.431759 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.432449 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.432588 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.433115 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:24.433276 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:24.433369 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.433546 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.433637 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260619 15:02:24.447643 2566 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.445110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37264 (local address 127.11.175.252:41039)
0619 15:02:24.445324 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.445331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.445906 (+ 575us) server_negotiation.cc:407] Connection header received
0619 15:02:24.446122 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.446129 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.446240 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.446355 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.447405 (+ 1050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37264: BlockingRecv error: recv got EOF from 127.0.0.1:37264 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 15:02:24.465386 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.457159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.11.175.253:37153)
0619 15:02:24.457448 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.457455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.458060 (+ 605us) server_negotiation.cc:407] Connection header received
0619 15:02:24.458320 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.458326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.458441 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.458551 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:24.459269 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.460632 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.462996 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.463823 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.463971 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.464552 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:24.464706 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:24.464801 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.464995 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.465086 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
W20260619 15:02:24.482415 2566 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.479773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37266 (local address 127.11.175.252:41039)
0619 15:02:24.480062 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.480069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.480748 (+ 679us) server_negotiation.cc:407] Connection header received
0619 15:02:24.480945 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.480953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.481071 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.481206 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.482151 (+ 945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37266: BlockingRecv error: recv got EOF from 127.0.0.1:37266 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
I20260619 15:02:24.502782 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.494584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.11.175.253:37153)
0619 15:02:24.494876 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.494884 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.495585 (+ 701us) server_negotiation.cc:407] Connection header received
0619 15:02:24.495839 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.495846 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.495948 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.496064 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:24.496698 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.498084 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.500379 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.501113 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.501256 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.501851 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:24.502013 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:24.502112 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.502320 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.502418 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
W20260619 15:02:24.513958 2566 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.511375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.11.175.252:41039)
0619 15:02:24.511598 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.511605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.512339 (+ 734us) server_negotiation.cc:407] Connection header received
0619 15:02:24.512592 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.512599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.512699 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.512810 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.513723 (+ 913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37272: BlockingRecv error: recv got EOF from 127.0.0.1:37272 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 15:02:24.541363 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.533890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41608 (local address 127.11.175.253:37153)
0619 15:02:24.534117 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.534126 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.534732 (+ 606us) server_negotiation.cc:407] Connection header received
0619 15:02:24.534968 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.534974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.535063 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.535165 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:24.535686 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.536861 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.538913 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.539557 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.539691 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.540310 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:24.540523 (+ 213us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:24.540613 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.540869 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.540967 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260619 15:02:24.552225 2566 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.549675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.11.175.252:41039)
0619 15:02:24.549942 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.549949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.550508 (+ 559us) server_negotiation.cc:407] Connection header received
0619 15:02:24.550776 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.550781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.550881 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.551005 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.551955 (+ 950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37286: BlockingRecv error: recv got EOF from 127.0.0.1:37286 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
I20260619 15:02:24.600663 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.592775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.11.175.253:37153)
0619 15:02:24.593053 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.593060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.593629 (+ 569us) server_negotiation.cc:407] Connection header received
0619 15:02:24.593969 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.593975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.594127 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.594245 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:24.594753 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.595949 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.598302 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.599117 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.599285 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.599773 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:24.599921 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:24.600031 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.600243 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.600344 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260619 15:02:24.614778 2566 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.612330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37290 (local address 127.11.175.252:41039)
0619 15:02:24.612513 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.612520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.613114 (+ 594us) server_negotiation.cc:407] Connection header received
0619 15:02:24.613284 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.613290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.613393 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.613515 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.614490 (+ 975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37290: BlockingRecv error: recv got EOF from 127.0.0.1:37290 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 15:02:24.688717 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.682317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41620 (local address 127.11.175.253:37153)
0619 15:02:24.682532 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.682538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.682936 (+ 398us) server_negotiation.cc:407] Connection header received
0619 15:02:24.683112 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.683118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.683210 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.683322 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:24.683818 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.684953 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.686695 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.687266 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.687458 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.687952 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:24.688085 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:24.688201 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.688333 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.688427 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260619 15:02:24.698473 2566 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.696272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37292 (local address 127.11.175.252:41039)
0619 15:02:24.696516 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.696523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.696831 (+ 308us) server_negotiation.cc:407] Connection header received
0619 15:02:24.697013 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.697020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.697133 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.697256 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.698217 (+ 961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37292: BlockingRecv error: recv got EOF from 127.0.0.1:37292 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 15:02:24.836652 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:24.830462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.11.175.253:37153)
0619 15:02:24.830807 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.830816 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.831053 (+ 237us) server_negotiation.cc:407] Connection header received
0619 15:02:24.831370 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.831375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.831468 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.831576 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:24.832138 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.833165 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.834709 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:24.835149 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:24.835312 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:24.835850 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:24.835988 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:24.836147 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:24.836352 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 15:02:24.836428 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260619 15:02:24.844784 2566 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 15:02:24.842928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37304 (local address 127.11.175.252:41039)
0619 15:02:24.843189 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:24.843195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:24.843463 (+ 268us) server_negotiation.cc:407] Connection header received
0619 15:02:24.843618 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:24.843622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:24.843703 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:24.843791 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 15:02:24.844531 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37304: BlockingRecv error: recv got EOF from 127.0.0.1:37304 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260619 15:02:25.097561 2607 catalog_manager.cc:1259] Loaded cluster ID: f240833aac104a1699df0bc48ecc9928
I20260619 15:02:25.097690 2607 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f: loading cluster ID for follower catalog manager: success
I20260619 15:02:25.100592 2607 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f: acquiring CA information for follower catalog manager: success
I20260619 15:02:25.103026 2607 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f03fd8ef3e8340fea14a7e0aca59953f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 15:02:25.110975 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:25.104902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.11.175.253:37153)
0619 15:02:25.105111 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:25.105120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:25.105397 (+ 277us) server_negotiation.cc:407] Connection header received
0619 15:02:25.105564 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:25.105570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:25.105663 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:25.105777 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:25.106194 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:25.107396 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:25.108904 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:25.109549 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:25.109701 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:25.110125 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:25.110276 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:25.110373 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:25.110562 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 15:02:25.110682 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 15:02:25.126830 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:25.118393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.11.175.252:41039)
0619 15:02:25.118590 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:25.118597 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:25.118872 (+ 275us) server_negotiation.cc:407] Connection header received
0619 15:02:25.119079 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:25.119083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:25.119233 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:25.119358 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:25.119889 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:25.120705 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:25.122699 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:25.123535 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:25.123711 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:25.124781 (+ 1070us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:25.125712 (+ 931us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:25.125979 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:25.126138 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 15:02:25.126315 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 15:02:25.161180 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:25.146370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.11.175.253:37153)
0619 15:02:25.146606 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:25.146612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:25.153627 (+ 7015us) server_negotiation.cc:407] Connection header received
0619 15:02:25.154159 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:25.154165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:25.154264 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:25.154331 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:25.155249 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:25.156383 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:25.158713 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:25.159372 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:25.159477 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:25.160287 (+ 810us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:25.160430 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:25.160509 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:25.160750 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 15:02:25.160853 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260619 15:02:25.168336 2743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:25.146563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.11.175.254:41529)
0619 15:02:25.158460 (+ 11897us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:25.158469 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:25.158497 (+ 28us) server_negotiation.cc:407] Connection header received
0619 15:02:25.158806 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:25.158816 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:25.159093 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:25.160242 (+ 1149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:25.160466 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:25.162308 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:25.164471 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:25.165097 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:25.165236 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:25.166910 (+ 1674us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:25.167324 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:25.167434 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:25.167870 (+ 436us) server_negotiation.cc:299] Negotiation successful
0619 15:02:25.167973 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11707,"thread_start_us":12091,"threads_started":1}
I20260619 15:02:25.169510 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 15:02:25.153215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37324 (local address 127.11.175.252:41039)
0619 15:02:25.153422 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 15:02:25.153429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 15:02:25.163316 (+ 9887us) server_negotiation.cc:407] Connection header received
0619 15:02:25.163478 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 15:02:25.163484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 15:02:25.163585 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 15:02:25.163694 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 15:02:25.164352 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:25.165251 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:25.167294 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 15:02:25.168161 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 15:02:25.168329 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 15:02:25.168809 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 15:02:25.168916 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 15:02:25.169023 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 15:02:25.169177 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 15:02:25.169260 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 15:02:25.174437 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 2343
I20260619 15:02:25.175925 2405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:02:25.292448 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 2343
W20260619 15:02:25.314026 2511 connection.cc:535] server connection from 127.0.0.1:37174 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 15:02:25.314944 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 2417
I20260619 15:02:25.316251 2479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:02:25.426515 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 2417
I20260619 15:02:25.449105 11967 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu pid 2492
I20260619 15:02:25.450281 2554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 15:02:25.574199 11967 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDoOfXv/build/asan/bin/kudu with pid 2492
2026-06-19T15:02:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4729 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9812 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (372541 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST